Feature extraction for machine learning-based intrusion detection in IoT networks

A large number of network security breaches in IoT networks have demonstrated the unreliability of current Network Intrusion Detection Systems(NIDSs).Consequently,network interruptions and loss of sensitive data have occurred,which led to an active research area for improving NIDS technologies.In an analysis of related works,it was observed that most researchers aim to obtain better classification results by using a set of untried combinations of Feature Reduction(FR)and Machine Learning(ML)techniques on NIDS datasets.However,these datasets are different in feature sets,attack types,and network design.Therefore,this paper aims to discover whether these techniques can be generalised across various datasets.Six ML models are utilised:a Deep Feed Forward(DFF),Convolutional Neural Network(CNN),Recurrent Neural Network(RNN),Decision Tree(DT),Logistic Regression(LR),and Naive Bayes(NB).The accuracy of three Feature Extraction(FE)algorithms is detected;Principal Component Analysis(PCA),Auto-encoder(AE),and Linear Discriminant Analysis(LDA),are evaluated using three benchmark datasets:UNSW-NB15,ToN-IoT and CSE-CIC-IDS2018.Although PCA and AE algorithms have been widely used,the determination of their optimal number of extracted dimensions has been overlooked.The results indicate that no clear FE method or ML model can achieve the best scores for all datasets.The optimal number of extracted dimensions has been identified for each dataset,and LDA degrades the performance of the ML models on two datasets.The variance is used to analyse the extracted dimensions of LDA and PCA.Finally,this paper concludes that the choice of datasets significantly alters the performance of the applied techniques.We believe that a universal(benchmark)feature set is needed to facilitate further advancement and progress of research in this field.

Anomaly Detection in Imbalanced Encrypted Traffic with Few Packet Metadata-Based Feature Extraction

In the IoT(Internet of Things)domain,the increased use of encryption protocols such as SSL/TLS,VPN(Virtual Private Network),and Tor has led to a rise in attacks leveraging encrypted traffic.While research on anomaly detection using AI(Artificial Intelligence)is actively progressing,the encrypted nature of the data poses challenges for labeling,resulting in data imbalance and biased feature extraction toward specific nodes.This study proposes a reconstruction error-based anomaly detection method using an autoencoder(AE)that utilizes packet metadata excluding specific node information.The proposed method omits biased packet metadata such as IP and Port and trains the detection model using only normal data,leveraging a small amount of packet metadata.This makes it well-suited for direct application in IoT environments due to its low resource consumption.In experiments comparing feature extraction methods for AE-based anomaly detection,we found that using flowbased features significantly improves accuracy,precision,F1 score,and AUC(Area Under the Receiver Operating Characteristic Curve)score compared to packet-based features.Additionally,for flow-based features,the proposed method showed a 30.17%increase in F1 score and improved false positive rates compared to Isolation Forest and OneClassSVM.Furthermore,the proposedmethod demonstrated a 32.43%higherAUCwhen using packet features and a 111.39%higher AUC when using flow features,compared to previously proposed oversampling methods.This study highlights the impact of feature extraction methods on attack detection in imbalanced,encrypted traffic environments and emphasizes that the one-class method using AE is more effective for attack detection and reducing false positives compared to traditional oversampling methods.

CMMCAN:Lightweight Feature Extraction and Matching Network for Endoscopic Images Based on Adaptive Attention

In minimally invasive surgery,endoscopes or laparoscopes equipped with miniature cameras and tools are used to enter the human body for therapeutic purposes through small incisions or natural cavities.However,in clinical operating environments,endoscopic images often suffer from challenges such as low texture,uneven illumination,and non-rigid structures,which affect feature observation and extraction.This can severely impact surgical navigation or clinical diagnosis due to missing feature points in endoscopic images,leading to treatment and postoperative recovery issues for patients.To address these challenges,this paper introduces,for the first time,a Cross-Channel Multi-Modal Adaptive Spatial Feature Fusion(ASFF)module based on the lightweight architecture of EfficientViT.Additionally,a novel lightweight feature extraction and matching network based on attention mechanism is proposed.This network dynamically adjusts attention weights for cross-modal information from grayscale images and optical flow images through a dual-branch Siamese network.It extracts static and dynamic information features ranging from low-level to high-level,and from local to global,ensuring robust feature extraction across different widths,noise levels,and blur scenarios.Global and local matching are performed through a multi-level cascaded attention mechanism,with cross-channel attention introduced to simultaneously extract low-level and high-level features.Extensive ablation experiments and comparative studies are conducted on the HyperKvasir,EAD,M2caiSeg,CVC-ClinicDB,and UCL synthetic datasets.Experimental results demonstrate that the proposed network improves upon the baseline EfficientViT-B3 model by 75.4%in accuracy(Acc),while also enhancing runtime performance and storage efficiency.When compared with the complex DenseDescriptor feature extraction network,the difference in Acc is less than 7.22%,and IoU calculation results on specific datasets outperform complex dense models.Furthermore,this method increases the F1 score by 33.2%and accelerates runtime by 70.2%.It is noteworthy that the speed of CMMCAN surpasses that of comparative lightweight models,with feature extraction and matching performance comparable to existing complex models but with faster speed and higher cost-effectiveness.

A three-dimensional feature extraction-based method for coal cleat characterization using X-ray μCT and its application to a Bowen Basin coal specimen

Cleats are the dominant micro-fracture network controlling the macro-mechanical behavior of coal.Improved understanding of the spatial characteristics of cleat networks is therefore important to the coal mining industry.Discrete fracture networks(DFNs)are increasingly used in engineering analyses to spatially model fractures at various scales.The reliability of coal DFNs largely depends on the confidence in the input cleat statistics.Estimates of these parameters can be made from image-based three-dimensional(3D)characterization of coal cleats using X-ray micro-computed tomography(m CT).One key step in this process,after cleat extraction,is the separation of individual cleats,without which the cleats are a connected network and statistics for different cleat sets cannot be measured.In this paper,a feature extraction-based image processing method is introduced to identify and separate distinct cleat groups from 3D X-ray m CT images.Kernels(filters)representing explicit cleat features of coal are built and cleat separation is successfully achieved by convolutional operations on 3D coal images.The new method is applied to a coal specimen with 80 mm in diameter and 100 mm in length acquired from an Anglo American Steelmaking Coal mine in the Bowen Basin,Queensland,Australia.It is demonstrated that the new method produces reliable cleat separation capable of defining individual cleats and preserving 3D topology after separation.Bedding-parallel fractures are also identified and separated,which has his-torically been challenging to delineate and rarely reported.A variety of cleat/fracture statistics is measured which not only can quantitatively characterize the cleat/fracture system but also can be used for DFN modeling.Finally,variability and heterogeneity with respect to the core axis are investigated.Significant heterogeneity is observed and suggests that the representative elementary volume(REV)of the cleat groups for engineering purposes may be a complex problem requiring careful consideration.

The Influence of Air Pollution Concentrations on Solar Irradiance Forecasting Using CNN-LSTM-mRMR Feature Extraction

Maintaining a steady power supply requires accurate forecasting of solar irradiance,since clean energy resources do not provide steady power.The existing forecasting studies have examined the limited effects of weather conditions on solar radiation such as temperature and precipitation utilizing convolutional neural network(CNN),but no comprehensive study has been conducted on concentrations of air pollutants along with weather conditions.This paper proposes a hybrid approach based on deep learning,expanding the feature set by adding new air pollution concentrations,and ranking these features to select and reduce their size to improve efficiency.In order to improve the accuracy of feature selection,a maximum-dependency and minimum-redundancy(mRMR)criterion is applied to the constructed feature space to identify and rank the features.The combination of air pollution data with weather conditions data has enabled the prediction of solar irradiance with a higher accuracy.An evaluation of the proposed approach is conducted in Istanbul over 12 months for 43791 discrete times,with the main purpose of analyzing air data,including particular matter(PM10 and PM25),carbon monoxide(CO),nitric oxide(NOX),nitrogen dioxide(NO_(2)),ozone(O₃),sulfur dioxide(SO_(2))using a CNN,a long short-term memory network(LSTM),and MRMR feature extraction.Compared with the benchmark models with root mean square error(RMSE)results of 76.2,60.3,41.3,32.4,there is a significant improvement with the RMSE result of 5.536.This hybrid model presented here offers high prediction accuracy,a wider feature set,and a novel approach based on air concentrations combined with weather conditions for solar irradiance prediction.

Weak Fault Feature Extraction of the Rotating Machinery Using Flexible Analytic Wavelet Transform and Nonlinear Quantum Permutation Entropy

Addressing the challenges posed by the nonlinear and non-stationary vibrations in rotating machinery,where weak fault characteristic signals hinder accurate fault state representation,we propose a novel feature extraction method that combines the Flexible Analytic Wavelet Transform(FAWT)with Nonlinear Quantum Permutation Entropy.FAWT,leveraging fractional orders and arbitrary scaling and translation factors,exhibits superior translational invariance and adjustable fundamental oscillatory characteristics.This flexibility enables FAWT to provide well-suited wavelet shapes,effectively matching subtle fault components and avoiding performance degradation associated with fixed frequency partitioning and low-oscillation bases in detecting weak faults.In our approach,gearbox vibration signals undergo FAWT to obtain sub-bands.Quantum theory is then introduced into permutation entropy to propose Nonlinear Quantum Permutation Entropy,a feature that more accurately characterizes the operational state of vibration simulation signals.The nonlinear quantum permutation entropy extracted from sub-bands is utilized to characterize the operating state of rotating machinery.A comprehensive analysis of vibration signals from rolling bearings and gearboxes validates the feasibility of the proposed method.Comparative assessments with parameters derived from traditional permutation entropy,sample entropy,wavelet transform(WT),and empirical mode decomposition(EMD)underscore the superior effectiveness of this approach in fault detection and classification for rotating machinery.

Spatial Distribution Feature Extraction Network for Open Set Recognition of Electromagnetic Signal

This paper proposes a novel open set recognition method,the Spatial Distribution Feature Extraction Network(SDFEN),to address the problem of electromagnetic signal recognition in an open environment.The spatial distribution feature extraction layer in SDFEN replaces convolutional output neural networks with the spatial distribution features that focus more on inter-sample information by incorporating class center vectors.The designed hybrid loss function considers both intra-class distance and inter-class distance,thereby enhancing the similarity among samples of the same class and increasing the dissimilarity between samples of different classes during training.Consequently,this method allows unknown classes to occupy a larger space in the feature space.This reduces the possibility of overlap with known class samples and makes the boundaries between known and unknown samples more distinct.Additionally,the feature comparator threshold can be used to reject unknown samples.For signal open set recognition,seven methods,including the proposed method,are applied to two kinds of electromagnetic signal data:modulation signal and real-world emitter.The experimental results demonstrate that the proposed method outperforms the other six methods overall in a simulated open environment.Specifically,compared to the state-of-the-art Openmax method,the novel method achieves up to 8.87%and 5.25%higher micro-F-measures,respectively.

Feature extraction and learning approaches for cancellable biometrics:A survey

Biometric recognition is a widely used technology for user authentication.In the application of this technology,biometric security and recognition accuracy are two important issues that should be considered.In terms of biometric security,cancellable biometrics is an effective technique for protecting biometric data.Regarding recognition accuracy,feature representation plays a significant role in the performance and reliability of cancellable biometric systems.How to design good feature representations for cancellable biometrics is a challenging topic that has attracted a great deal of attention from the computer vision community,especially from researchers of cancellable biometrics.Feature extraction and learning in cancellable biometrics is to find suitable feature representations with a view to achieving satisfactory recognition performance,while the privacy of biometric data is protected.This survey informs the progress,trend and challenges of feature extraction and learning for cancellable biometrics,thus shedding light on the latest developments and future research of this area.

Feature extraction and damage alarming using time series analysis

Aiming at the problem of on-line damage diagnosis in structural health monitoring （SHM）, an algorithm of feature extraction and damage alarming based on auto-regressive moving-average （ARMA） time series analysis is presented. The monitoring data were first modeled as ARMA models, while a principalcomponent matrix derived from the AR coefficients of these models was utilized to establish the Mahalanobisdistance criterion functions. Then, a new damage-sensitive feature index DDSF is proposed. A hypothesis test involving the t-test method is further applied to obtain a decision of damage alarming as the mean value of DDSF had significantly changed after damage. The numerical results of a three-span-girder model shows that the defined index is sensitive to subtle structural damage, and the proposed algorithm can be applied to the on-line damage alarming in SHM.

Feature Extraction of Stored-grain Insects Based on Ant Colony Optimization and Support Vector Machine Algorithm

[Objective] The aim was to study the feature extraction of stored-grain insects based on ant colony optimization and support vector machine algorithm, and to explore the feasibility of the feature extraction of stored-grain insects. [Method] Through the analysis of feature extraction in the image recognition of the stored-grain insects, the recognition accuracy of the cross-validation training model in support vector machine （SVM） algorithm was taken as an important factor of the evaluation principle of feature extraction of stored-grain insects. The ant colony optimization （ACO） algorithm was applied to the automatic feature extraction of stored-grain insects. [Result] The algorithm extracted the optimal feature subspace of seven features from the 17 morphological features, including area and perimeter. The ninety image samples of the stored-grain insects were automatically recognized by the optimized SVM classifier, and the recognition accuracy was over 95%. [Conclusion] The experiment shows that the application of ant colony optimization to the feature extraction of grain insects is practical and feasible.

Human Ear Image Recognition Method Using PCA and Fisherface Complementary Double Feature Extraction

Ear recognition is a new kind of biometric identification technology now.Feature extraction is a key step in pattern recognition technology,which determines the accuracy of classification results.The method of single feature extraction can achieve high recognition rate under certain conditions,but the use of double feature extraction can overcome the limitation of single feature extraction.In order to improve the accuracy of classification results,this paper proposes a new method,that is,the method of complementary double feature extraction based on Principal Component Analysis(PCA)and Fisherface,and we apply it to human ear image recognition.The experiment was carried out on the ear image library provided by the University of Science and Technology Beijing.The results show that the ear recognition rate of the proposed method is significantly higher than the single feature extraction using PCA,Fisherface,or Independent component analysis(ICA)alone.

Feature Extraction and Recognition for Rolling Element Bearing Fault Utilizing Short-Time Fourier Transform and Non-negative Matrix Factorization

Due to the non-stationary characteristics of vibration signals acquired from rolling element bearing fault, thc time-frequency analysis is often applied to describe the local information of these unstable signals smartly. However, it is difficult to classitythe high dimensional feature matrix directly because of too large dimensions for many classifiers. This paper combines the concepts of time-frequency distribution（TFD） with non-negative matrix factorization（NMF）, and proposes a novel TFD matrix factorization method to enhance representation and identification of bearing fault. Throughout this method, the TFD of a vibration signal is firstly accomplished to describe the localized faults with short-time Fourier transform（STFT）. Then, the supervised NMF mapping is adopted to extract the fault features from TFD. Meanwhile, the fault samples can be clustered and recognized automatically by using the clustering property of NMF. The proposed method takes advantages of the NMF in the parts-based representation and the adaptive clustering. The localized fault features of interest can be extracted as well. To evaluate the performance of the proposed method, the 9 kinds of the bearing fault on a test bench is performed. The proposed method can effectively identify the fault severity and different fault types. Moreover, in comparison with the artificial neural network（ANN）, NMF yields 99.3% mean accuracy which is much superior to ANN. This research presents a simple and practical resolution for the fault diagnosis problem of rolling element bearing in high dimensional feature space.

Auditory-model-based Feature Extraction Method for Mechanical Faults Diagnosis

It is well known that the human auditory system possesses remarkable capabilities to analyze and identify signals. Therefore, it would be significant to build an auditory model based on the mechanism of human auditory systems, which may improve the effects of mechanical signal analysis and enrich the methods of mechanical faults features extraction. However the existing methods are all based on explicit senses of mathematics or physics, and have some shortages on distinguishing different faults, stability, and suppressing the disturbance noise, etc. For the purpose of improving the performances of the work of feature extraction, an auditory model, early auditory（EA） model, is introduced for the first time. This auditory model transforms time domain signal into auditory spectrum via bandpass filtering, nonlinear compressing, and lateral inhibiting by simulating the principle of the human auditory system. The EA model is developed with the Gammatone filterbank as the basilar membrane. According to the characteristics of vibration signals, a method is proposed for determining the parameter of inner hair cells model of EA model. The performance of EA model is evaluated through experiments on four rotor faults, including misalignment, rotor-to-stator rubbing, oil film whirl, and pedestal looseness. The results show that the auditory spectrum, output of EA model, can effectively distinguish different faults with satisfactory stability and has the ability to suppress the disturbance noise. Then, it is feasible to apply auditory model, as a new method, to the feature extraction for mechanical faults diagnosis with effect.

Feature Extraction Method Based on Pseudo-Wigner-Ville Distribution for Rotational Machinery in Variable Operating Conditions

In the case of fault diagnosis for roller bearings, the conventional diagnosis approaches by using the time interval of energy impacts in time-frequency distribution or the pass-frequencies are based on the assumption that machinery operates under a constant rotational speed. However, when the rotational speed varies in the broader range, the pass-frequencies vary with the change of rotational speed and bearing faults cannot be identified by the interval of impacts. Researches related to automatic diagnosis for rotational machinery in variable operating conditions were quite few. A novel automatic feature extraction method is proposed based on a pseudo-Wigner-Ville distribution （PWVD） and an extraction of symptom parameter （SP）. An extraction method for instantaneous feature spectrum is presented using the relative crossing information （RCI） and sequential inference approach, by which the feature spectrum from time-frequency distribution can be automatically, sequentially extracted. The SPs are considered in the frequency domain using the extracted feature spectrum to identify among the conditions of a machine. A method to obtain the synthetic symptom parameter is also proposed by the least squares mapping （LSM） technique for increasing the diagnosis sensitivity of SP. Practical examples of diagnosis for bearings are given in order to verify the effectiveness of the proposed method. The verification results show that the features of bearing faults, such as the outer-race, inner-race and roller element defects have been effectively extracted, and the proposed method can be used for condition diagnosis of a machine under the variable rotational speed.

Application of multi-scale feature extraction to surface defect classification of hot-rolled steels

Feature extraction is essential to the classification of surface defect images. The defects of hot-rolled steels distribute in different directions. Therefore, the methods of multi-scale geometric analysis （MGA） were employed to decompose the image into several directional subba^ds at several scales. Then, the statistical features of each subband were calculated to produce a high-dimensional feature vector, which was reduced to a lower-dimensional vector by graph embedding algorithms. Finally, support vector machine （SVM） was used for defect classification. The multi-scale feature extraction method was implemented via curvelet transform and kernel locality preserving projections （KLPP）. Experiment results show that the proposed method is effective for classifying the surface defects of hot-rolled steels and the total classification rate is up to 97.33%.

SIGNAL FEATURE EXTRACTION BASED UPON INDEPENDENT COMPONENT ANALYSIS AND WAVELET TRANSFORM

It is an important precondition for machine fault diagnosis that vibrationsignal can be extracted effectively. Based on the characteristic of noise interfused during thecourse of sampling vibration signal, independent component analysis (ICA) method is combined withwavelet to de-noise. Firstly, The sampled signal can be separated with ICA, then the function offrequency band chosen with multi-resolution wavelet transform can be used to judge whether thestochastic disturbance singular signal is interfused. By these ways, the vibration signals can beextracted effectively, which provides favorable condition for subsequent feature detection ofvibration signal and fault diagnosis.

FEATURE EXTRACTION OF VIBRATION SIGNALS BASED ON WAVELET PACKET TRANSFORM

A method is proposed for the analysis of vibration signals from components ofrotating machines, based on the wavelet packet transformation (WPT) and the underlying physicalconcepts of modulation mechanism. The method provides a finer analysis and better time-frequencylocalization capabilities than any other analysis methods. Both details and approximations are splitinto finer components and result in better-localized frequency ranges corresponding to each node ofa wavelet packet tree. For the punpose of feature extraction, a hard threshold is given and theenergy of the coefficients above the threshold is used, as a criterion for the selection of the bestvector. The feature extraction of a vibration signal is accomplished by computing thereconstruction signal and its spectrum. When applied to a rolling bear vibration signal featureextraction, the proposed method can lead to be very effective.

Research on Feature Extraction of Remnant Particles of Aerospace Relays

The existence of remnant particles, which significantly reduce the reliability of relays, is a serious problem for aerospace relays. The traditional method for detecting remnant particles-particle impact noise detection （PIND）-can be used merely to detect the existence of the particle; it is not able to provide any information about the particles＇ material. However, information on the material of the particles is very helpful for analyzing the causes of remnants. By analyzing the output acoustic signals from a PIND tester, this paper proposes three feature extraction methods： unit energy average pulse durative time, shape parameter of signal power spectral density （PSD）, and pulse linear predictive coding coefficient sequence. These methods allow identified remnants to be classified into four categories based on their material. Furthermore, we prove the validity of this new method by processing P1ND signals from actual tests.

A Multi-model Approach for Soft Sensor Development Based on Feature Extraction Using Weighted Kernel Fisher Criterion

Multi-model approach can significantly improve the prediction performance of soft sensors in the process with multiple operational conditions.However,traditional clustering algorithms may result in overlapping phenomenon in subclasses,so that edge classes and outliers cannot be effectively dealt with and the modeling result is not satisfactory.In order to solve these problems,a new feature extraction method based on weighted kernel Fisher criterion is presented to improve the clustering accuracy,in which feature mapping is adopted to bring the edge classes and outliers closer to other normal subclasses.Furthermore,the classified data are used to develop a multiple model based on support vector machine.The proposed method is applied to a bisphenol A production process for prediction of the quality index.The simulation results demonstrate its ability in improving the data classification and the prediction performance of the soft sensor.

Fractional Envelope Analysis for Rolling Element Bearing Weak Fault Feature Extraction

The bearing weak fault feature extraction is crucial to mechanical fault diagnosis and machine condition monitoring. Envelope analysis based on Hilbert transform has been widely used in bearing fault feature extraction. A generalization of the Hilbert transform, the fractional Hilbert transform is defined in the frequency domain, it is based upon the modification of spatial filter with a fractional parameter, and it can be used to construct a new kind of fractional analytic signal. By performing spectrum analysis on the fractional envelope signal, the fractional envelope spectrum can be obtained. When weak faults occur in a bearing, some of the characteristic frequencies will clearly appear in the fractional envelope spectrum. These characteristic frequencies can be used for bearing weak fault feature extraction. The effectiveness of the proposed method is verified through simulation signal and experiment data. © 2017 Chinese Association of Automation.