IDS-INT:Intrusion detection system using transformer-based transfer learning for imbalanced network traffic

A network intrusion detection system is critical for cyber security against llegitimate attacks.In terms of feature perspectives,network traffic may include a variety of elements such as attack reference,attack type,a subcategory of attack,host information,malicious scripts,etc.In terms of network perspectives,network traffic may contain an imbalanced number of harmful attacks when compared to normal traffic.It is challenging to identify a specific attack due to complex features and data imbalance issues.To address these issues,this paper proposes an Intrusion Detection System using transformer-based transfer learning for Imbalanced Network Traffic(IDS-INT).IDS-INT uses transformer-based transfer learning to learn feature interactions in both network feature representation and imbalanced data.First,detailed information about each type of attack is gathered from network interaction descriptions,which include network nodes,attack type,reference,host information,etc.Second,the transformer-based transfer learning approach is developed to learn detailed feature representation using their semantic anchors.Third,the Synthetic Minority Oversampling Technique(SMOTE)is implemented to balance abnormal traffic and detect minority attacks.Fourth,the Convolution Neural Network(CNN)model is designed to extract deep features from the balanced network traffic.Finally,the hybrid approach of the CNN-Long Short-Term Memory(CNN-LSTM)model is developed to detect different types of attacks from the deep features.Detailed experiments are conducted to test the proposed approach using three standard datasets,i.e.,UNsWNB15,CIC-IDS2017,and NSL-KDD.An explainable AI approach is implemented to interpret the proposed method and develop a trustable model.

Feature extraction for machine learning-based intrusion detection in IoT networks

A large number of network security breaches in IoT networks have demonstrated the unreliability of current Network Intrusion Detection Systems(NIDSs).Consequently,network interruptions and loss of sensitive data have occurred,which led to an active research area for improving NIDS technologies.In an analysis of related works,it was observed that most researchers aim to obtain better classification results by using a set of untried combinations of Feature Reduction(FR)and Machine Learning(ML)techniques on NIDS datasets.However,these datasets are different in feature sets,attack types,and network design.Therefore,this paper aims to discover whether these techniques can be generalised across various datasets.Six ML models are utilised:a Deep Feed Forward(DFF),Convolutional Neural Network(CNN),Recurrent Neural Network(RNN),Decision Tree(DT),Logistic Regression(LR),and Naive Bayes(NB).The accuracy of three Feature Extraction(FE)algorithms is detected;Principal Component Analysis(PCA),Auto-encoder(AE),and Linear Discriminant Analysis(LDA),are evaluated using three benchmark datasets:UNSW-NB15,ToN-IoT and CSE-CIC-IDS2018.Although PCA and AE algorithms have been widely used,the determination of their optimal number of extracted dimensions has been overlooked.The results indicate that no clear FE method or ML model can achieve the best scores for all datasets.The optimal number of extracted dimensions has been identified for each dataset,and LDA degrades the performance of the ML models on two datasets.The variance is used to analyse the extracted dimensions of LDA and PCA.Finally,this paper concludes that the choice of datasets significantly alters the performance of the applied techniques.We believe that a universal(benchmark)feature set is needed to facilitate further advancement and progress of research in this field.

Network Intrusion Traffic Detection Based on Feature Extraction

With the increasing dimensionality of network traffic,extracting effective traffic features and improving the identification accuracy of different intrusion traffic have become critical in intrusion detection systems(IDS).However,both unsupervised and semisupervised anomalous traffic detection methods suffer from the drawback of ignoring potential correlations between features,resulting in an analysis that is not an optimal set.Therefore,in order to extract more representative traffic features as well as to improve the accuracy of traffic identification,this paper proposes a feature dimensionality reduction method combining principal component analysis and Hotelling’s T^(2) and a multilayer convolutional bidirectional long short-term memory(MSC_BiLSTM)classifier model for network traffic intrusion detection.This method reduces the parameters and redundancy of the model by feature extraction and extracts the dependent features between the data by a bidirectional long short-term memory(BiLSTM)network,which fully considers the influence between the before and after features.The network traffic is first characteristically downscaled by principal component analysis(PCA),and then the downscaled principal components are used as input to Hotelling’s T^(2) to compare the differences between groups.For datasets with outliers,Hotelling’s T^(2) can help identify the groups where the outliers are located and quantitatively measure the extent of the outliers.Finally,a multilayer convolutional neural network and a BiLSTM network are used to extract the spatial and temporal features of network traffic data.The empirical consequences exhibit that the suggested approach in this manuscript attains superior outcomes in precision,recall and F1-score juxtaposed with the prevailing techniques.The results show that the intrusion detection accuracy,precision,and F1-score of the proposed MSC_BiLSTM model for the CIC-IDS 2017 dataset are 98.71%,95.97%,and 90.22%.

Machine Learning Models for Heterogenous Network Security Anomaly Detection

The increasing amount and intricacy of network traffic in the modern digital era have worsened the difficulty of identifying abnormal behaviours that may indicate potential security breaches or operational interruptions. Conventional detection approaches face challenges in keeping up with the ever-changing strategies of cyber-attacks, resulting in heightened susceptibility and significant harm to network infrastructures. In order to tackle this urgent issue, this project focused on developing an effective anomaly detection system that utilizes Machine Learning technology. The suggested model utilizes contemporary machine learning algorithms and frameworks to autonomously detect deviations from typical network behaviour. It promptly identifies anomalous activities that may indicate security breaches or performance difficulties. The solution entails a multi-faceted approach encompassing data collection, preprocessing, feature engineering, model training, and evaluation. By utilizing machine learning methods, the model is trained on a wide range of datasets that include both regular and abnormal network traffic patterns. This training ensures that the model can adapt to numerous scenarios. The main priority is to ensure that the system is functional and efficient, with a particular emphasis on reducing false positives to avoid unwanted alerts. Additionally, efforts are directed on improving anomaly detection accuracy so that the model can consistently distinguish between potentially harmful and benign activity. This project aims to greatly strengthen network security by addressing emerging cyber threats and improving their resilience and reliability.

Dis-NDVW: Distributed Network Asset Detection and Vulnerability Warning Platform

With the rapid development of Internet technology,the issues of network asset detection and vulnerability warning have become hot topics of concern in the industry.However,most existing detection tools operate in a single-node mode and cannot parallelly process large-scale tasks,which cannot meet the current needs of the industry.To address the above issues,this paper proposes a distributed network asset detection and vulnerability warning platform(Dis-NDVW)based on distributed systems and multiple detection tools.Specifically,this paper proposes a distributed message sub-scription and publication system based on Zookeeper and Kafka,which endows Dis-NDVW with the ability to parallelly process large-scale tasks.Meanwhile,Dis-NDVW combines the RangeAssignor,RoundRobinAssignor,and StickyAssignor algorithms to achieve load balancing of task nodes in a distributed detection cluster.In terms of a large-scale task processing strategy,this paper proposes a task partitioning method based on First-In-First-Out(FIFO)queue.This method realizes the parallel operation of task producers and task consumers by dividing pending tasks into different queues according to task types.To ensure the data reliability of the task cluster,Dis-NDVW provides a redundant storage strategy for master-slave partition replicas.In terms of distributed storage,Dis-NDVW utilizes a distributed elastic storage service based on ElasticSearch to achieve distributed storage and efficient retrieval of big data.Experimental verification shows that Dis-NDVW can better meet the basic requirements of ultra-large-scale detection tasks.

Network Intrusion Detection in Internet of Blended Environment Using Ensemble of Heterogeneous Autoencoders(E-HAE)

Contemporary attackers,mainly motivated by financial gain,consistently devise sophisticated penetration techniques to access important information or data.The growing use of Internet of Things(IoT)technology in the contemporary convergence environment to connect to corporate networks and cloud-based applications only worsens this situation,as it facilitates multiple new attack vectors to emerge effortlessly.As such,existing intrusion detection systems suffer from performance degradation mainly because of insufficient considerations and poorly modeled detection systems.To address this problem,we designed a blended threat detection approach,considering the possible impact and dimensionality of new attack surfaces due to the aforementioned convergence.We collectively refer to the convergence of different technology sectors as the internet of blended environment.The proposed approach encompasses an ensemble of heterogeneous probabilistic autoencoders that leverage the corresponding advantages of a convolutional variational autoencoder and long short-term memory variational autoencoder.An extensive experimental analysis conducted on the TON_IoT dataset demonstrated 96.02%detection accuracy.Furthermore,performance of the proposed approach was compared with various single model(autoencoder)-based network intrusion detection approaches:autoencoder,variational autoencoder,convolutional variational autoencoder,and long short-term memory variational autoencoder.The proposed model outperformed all compared models,demonstrating F1-score improvements of 4.99%,2.25%,1.92%,and 3.69%,respectively.

A Time Series Intrusion Detection Method Based on SSAE,TCN and Bi-LSTM

In the fast-evolving landscape of digital networks,the incidence of network intrusions has escalated alarmingly.Simultaneously,the crucial role of time series data in intrusion detection remains largely underappreciated,with most systems failing to capture the time-bound nuances of network traffic.This leads to compromised detection accuracy and overlooked temporal patterns.Addressing this gap,we introduce a novel SSAE-TCN-BiLSTM(STL)model that integrates time series analysis,significantly enhancing detection capabilities.Our approach reduces feature dimensionalitywith a Stacked Sparse Autoencoder(SSAE)and extracts temporally relevant features through a Temporal Convolutional Network(TCN)and Bidirectional Long Short-term Memory Network(Bi-LSTM).By meticulously adjusting time steps,we underscore the significance of temporal data in bolstering detection accuracy.On the UNSW-NB15 dataset,ourmodel achieved an F1-score of 99.49%,Accuracy of 99.43%,Precision of 99.38%,Recall of 99.60%,and an inference time of 4.24 s.For the CICDS2017 dataset,we recorded an F1-score of 99.53%,Accuracy of 99.62%,Precision of 99.27%,Recall of 99.79%,and an inference time of 5.72 s.These findings not only confirm the STL model’s superior performance but also its operational efficiency,underpinning its significance in real-world cybersecurity scenarios where rapid response is paramount.Our contribution represents a significant advance in cybersecurity,proposing a model that excels in accuracy and adaptability to the dynamic nature of network traffic,setting a new benchmark for intrusion detection systems.

Detection Situation and Improvement of Lightning Monitoring Network in Guizhou Province

Based on lightning location data of lightning monitoring network in Guizhou Province in recent eight years,the effective detection radius of a station and the effective detection range of lightning monitoring network in Guizhou Province were analyzed. The results show that the effective detection radius of a lightning monitoring sub-station in Guizhou Province is 160 km; some counties in the southwest,northwest and northeast of Guizhou were not detected. To improve the detector efficiency of lightning monitoring network in Guizhou Province,it is suggested that nine sub-stations should be built in Weining,Shuicheng,Qinglong,Pingtang,Rongjiang,Yuping,Songtao,Tongren and Renhuai,so that the effective detection efficiency will reach more than 95%.

FEW-NNN: A Fuzzy Entropy Weighted Natural Nearest Neighbor Method for Flow-Based Network Traffic Attack Detection

Attacks such as APT usually hide communication data in massive legitimate network traffic, and mining structurally complex and latent relationships among flow-based network traffic to detect attacks has become the focus of many initiatives. Effectively analyzing massive network security data with high dimensions for suspicious flow diagnosis is a huge challenge. In addition, the uneven distribution of network traffic does not fully reflect the differences of class sample features, resulting in the low accuracy of attack detection. To solve these problems, a novel approach called the fuzzy entropy weighted natural nearest neighbor(FEW-NNN) method is proposed to enhance the accuracy and efficiency of flowbased network traffic attack detection. First, the FEW-NNN method uses the Fisher score and deep graph feature learning algorithm to remove unimportant features and reduce the data dimension. Then, according to the proposed natural nearest neighbor searching algorithm(NNN_Searching), the density of data points, each class center and the smallest enclosing sphere radius are determined correspondingly. Finally, a fuzzy entropy weighted KNN classification method based on affinity is proposed, which mainly includes the following three steps: 1、 the feature weights of samples are calculated based on fuzzy entropy values, 2、 the fuzzy memberships of samples are determined based on affinity among samples, and 3、 K-neighbors are selected according to the class-conditional weighted Euclidean distance, the fuzzy membership value of the testing sample is calculated based on the membership of k-neighbors, and then all testing samples are classified according to the fuzzy membership value of the samples belonging to each class;that is, the attack type is determined. The method has been applied to the problem of attack detection and validated based on the famous KDD99 and CICIDS-2017 datasets. From the experimental results shown in this paper, it is observed that the FEW-NNN method improves the accuracy and efficiency of flow-based network traffic attack detection.

Using Object Detection Network for Malware Detection and Identification in Network Traffic Packets

In recent years,the number of exposed vulnerabilities has grown rapidly and more and more attacks occurred to intrude on the target computers using these vulnerabilities such as different malware.Malware detection has attracted more attention and still faces severe challenges.As malware detection based traditional machine learning relies on exports’experience to design efficient features to distinguish different malware,it causes bottleneck on feature engineer and is also time-consuming to find efficient features.Due to its promising ability in automatically proposing and selecting significant features,deep learning has gradually become a research hotspot.In this paper,aiming to detect the malicious payload and identify their categories with high accuracy,we proposed a packet-based malicious payload detection and identification algorithm based on object detection deep learning network.A dataset of malicious payload on code execution vulnerability has been constructed under the Metasploit framework and used to evaluate the performance of the proposed malware detection and identification algorithm.The experimental results demonstrated that the proposed object detection network can efficiently find and identify malicious payloads with high accuracy.

A Novel Immune System Model and Its Application to Network Intrusion Detection

Based on analyzing the techniques and architecture of existing network Intrusion Detection System (IDS), and probing into the fundament of Immune System (IS), a novel immune model is presented and applied to network IDS, which is helpful to design an effective IDS. Besides, this paper suggests a scheme to represent the self profile of network. And an automated self profile extraction algorithm is provided to extract self profile from packets. The experimental results prove validity of the scheme and algorithm, which is the foundation of the immune model.

Clustering-based label estimation for network anomaly detection

A substantial body of work has been done to identify network anomalies using supervised and unsupervised learning techniques with their unique strengths and weaknesses.In this work,we propose a new approach that takes advantage of both worlds of unsupervised and supervised learnings.The main objective of the proposed approach is to enable supervised anomaly detection without the provision of the associated labels by users.To this end,we estimate the labels of each connection in the training phase using clustering.The“estimated”labels are then utilized to establish a supervised learning model for the subsequent classification of connections in the testing stage.We set up a new property that defines anomalies in the context of network anomaly detection to improve the quality of estimated labels.Through our extensive experiments with a public dataset(NSL-KDD),we will prove that the proposed method can achieve performance comparable to one with the “original”labels provided in the dataset.We also introduce two heuristic functions that minimize the impact of the randomness of clustering to improve the overall quality of the estimated labels.

Anomaly Detection of Complex Networks Based on Intuitionistic Fuzzy Set Ensemble

Ensemble learning for anomaly detection of data structured into a complex network has been barely studied due to the inconsistent performance of complex network characteristics and the lack of inherent objective function. We propose the intuitionistic fuzzy set（IFS）-based anomaly detection, a new two-phase ensemble method for anomaly detection based on IFS, and apply it to the abnormal behavior detection problem in temporal complex networks.Firstly, it constructs the IFS of a single network characteristic, which quantifies the degree of membership,non-membership and hesitation of each network characteristic to the defined linguistic variables so that makes the unuseful or noise characteristics become part of the detection. To build an objective intuitionistic fuzzy relationship, we propose a Gaussian distribution-based membership function which gives a variable hesitation degree. Then, for the fuzzification of multiple network characteristics, the intuitionistic fuzzy weighted geometric operator is adopted to fuse multiple IFSs and to avoid the inconsistence of multiple characteristics. Finally, the score function and precision function are used to sort the fused IFS. Finally, we carry out extensive experiments on several complex network datasets for anomaly detection, and the results demonstrate the superiority of our method to state-of-the-art approaches, validating the effectiveness of our method.

An Optimized and Hybrid Framework for Image Processing Based Network Intrusion Detection System

The network infrastructure has evolved rapidly due to the everincreasing volume of users and data.The massive number of online devices and users has forced the network to transform and facilitate the operational necessities of consumers.Among these necessities,network security is of prime significance.Network intrusion detection systems(NIDS)are among the most suitable approaches to detect anomalies and assaults on a network.However,keeping up with the network security requirements is quite challenging due to the constant mutation in attack patterns by the intruders.This paper presents an effective and prevalent framework for NIDS by merging image processing with convolution neural networks(CNN).The proposed framework first converts non-image data from network traffic into images and then further enhances those images by using the Gabor filter.The images are then classified using a CNN classifier.To assess the efficacy of the recommended method,four benchmark datasets i.e.,CSE-CIC-IDS2018,CIC-IDS-2017,ISCX-IDS 2012,and NSL-KDD were used.The proposed approach showed higher precision in contrast with the recent work on the mentioned datasets.Further,the proposed method is compared with the recent well-known image processing methods for NIDS.

A Step-Based Deep Learning Approach for Network Intrusion Detection

In the network security field,the network intrusion detection system(NIDS)is considered one of the critical issues in the detection accuracy andmissed detection rate.In this paper,amethod of two-step network intrusion detection on the basis of GoogLeNet Inception and deep convolutional neural networks(CNNs)models is proposed.The proposed method used the GoogLeNet Inception model to identify the network packets’binary problem.Subsequently,the characteristics of the packets’raw data and the traffic features are extracted.The CNNs model is also used to identify the multiclass intrusions by the network packets’features.In the experimental results,the proposed method shows an improvement in the identification accuracy,where it achieves up to 99.63%.In addition,the missed detection rate is reduced to be 0.1%.The results prove the high performance of the proposed method in enhancing the NIDS’s reliability.

Improved Ant Colony Optimization and Machine Learning Based Ensemble Intrusion Detection Model

Internet of things(IOT)possess cultural,commercial and social effect in life in the future.The nodes which are participating in IOT network are basi-cally attracted by the cyber-attack targets.Attack and identification of anomalies in IoT infrastructure is a growing problem in the IoT domain.Machine Learning Based Ensemble Intrusion Detection(MLEID)method is applied in order to resolve the drawback by minimizing malicious actions in related botnet attacks on Message Queue Telemetry Transport(MQTT)and Hyper-Text Transfer Proto-col(HTTP)protocols.The proposed work has two significant contributions which are a selection of features and detection of attacks.New features are chosen from Improved Ant Colony Optimization(IACO)in the feature selection,and then the detection of attacks is carried out based on a combination of their possible proper-ties.The IACO approach is focused on defining the attacker’s important features against HTTP and MQTT.In the IACO algorithm,the constant factor is calculated against HTTP and MQTT based on the mean function for each element.Attack detection,the performance of several machine learning models are Distance Deci-sion Tree(DDT),Adaptive Neuro-Fuzzy Inference System(ANFIS)and Mahala-nobis Distance Support Vector Machine(MDSVM)were compared with predicting accurate attacks on the IoT network.The outcomes of these classifiers are combined into the ensemble model.The proposed MLEID strategy has effec-tively established malicious incidents.The UNSW-NB15 dataset is used to test the MLEID technique using data from simulated IoT sensors.Besides,the pro-posed MLEID technique has a greater detection rate and an inferior rate of false-positive compared to other conventional techniques.

Intrusion detection systems for wireless sensor networks using computational intelligence techniques

Network Intrusion Detection Systems(NIDS)are utilized to find hostile network connections.This can be accom-plished by looking at traffic network activity,but it takes a lot of work.The NIDS heavily utilizes approaches for data extraction and machine learning to find anomalies.In terms of feature selection,NIDS is far more effective.This is accurate since anomaly identification uses a number of time-consuming features.Because of this,the feature selec-tion method influences how long it takes to analyze movement patterns and how clear it is.The goal of the study is to provide NIDS with an attribute selection approach.PSO has been used for that purpose.The Network Intrusion Detection System that is being developed will be able to identify any malicious activity in the network or any unusual behavior in the network,allowing the identification of the illegal activities and safeguarding the enormous amounts of confidential data belonging to the customers from being compromised.In the research,datasets were produced utilising both a network infrastructure and a simulation network.Wireshark is used to gather data packets whereas Cisco Packet Tracer is used to build a network in a simulated environment.Additionally,a physical network consisting of six node MCUs connected to a laptop and a mobile hotspot,has been built and communication packets are being recorded using the Wireshark tool.To train several machine learning models,all the datasets that were gatheredcre-ated datasets from our own studies as well as some common datasets like NSDL and UNSW acquired from Kaggle-were employed.Additionally,PsO,which is an optimization method,has been used with these ML algorithms for feature selection.In the research,KNN,decision trees,and ANN have all been combined with PSO for a specific case study.And it was found demonstrated the classification methods PSO+ANN outperformed PSO+KNN and PSO+DT in this case study.

SDN环境下基于CNN-BiLSTM的入侵检测研究

软件定义网络(SDN)是一种将控制层和数据层分离的新型网络架构,在实现网络集中管理和可编程性的同时也面临易受到入侵攻击的问题。针对此问题设计了检测防御机制。利用深度学习算法,对数据集进行处理后,融合卷积神经网络(CNN)和双向长短期记忆网络(BiLSTM),设计了CNN-BiLSTM模型检测攻击,利用SDN可编程性设计了防御机制,搭建基于SDN的网络平台进行仿真实验。实验结果表明,所设计方法相较传统检测方法可更准确检测出入侵流量,并在检测出后有效实现了防御功能。

FMSA:a meta-learning framework-based fast model stealing attack technique against intelligent network intrusion detection systems

Intrusion detection systems are increasingly using machine learning.While machine learning has shown excellent performance in identifying malicious traffic,it may increase the risk of privacy leakage.This paper focuses on imple-menting a model stealing attack on intrusion detection systems.Existing model stealing attacks are hard to imple-ment in practical network environments,as they either need private data of the victim dataset or frequent access to the victim model.In this paper,we propose a novel solution called Fast Model Stealing Attack(FMSA)to address the problem in the field of model stealing attacks.We also highlight the risks of using ML-NIDS in network security.First,meta-learning frameworks are introduced into the model stealing algorithm to clone the victim model in a black-box state.Then,the number of accesses to the target model is used as an optimization term,resulting in minimal queries to achieve model stealing.Finally,adversarial training is used to simulate the data distribution of the target model and achieve the recovery of privacy data.Through experiments on multiple public datasets,compared to existing state-of-the-art algorithms,FMSA reduces the number of accesses to the target model and improves the accuracy of the clone model on the test dataset to 88.9%and the similarity with the target model to 90.1%.We can demonstrate the successful execution of model stealing attacks on the ML-NIDS system even with protective measures in place to limit the number of anomalous queries.

Fusion network for small target detection based on YOLO and attention mechanism

Target detection is an important task in computer vision research, and such an anomaly detection and the topic of small target detection task is more concerned. However, there are still some problems in this kind of researches, such as small target detection in complex environments is susceptible to background interference and poor detection results. To solve these issues, this study proposes a method which introduces the attention mechanism into the you only look once(YOLO) network. In addition, the amateur-produced mask dataset was created and experiments were conducted. The results showed that the detection effect of the proposed mothed is much better.