Anomalous node detection in attributed social networks using dual variational autoencoder with generative adversarial networks

Many types of real-world information systems, including social media and e-commerce platforms, can be modelled by means of attribute-rich, connected networks. The goal of anomaly detection in artificial intelligence is to identify illustrations that deviate significantly from the main distribution of data or that differ from known cases. Anomalous nodes in node-attributed networks can be identified with greater precision if both graph and node attributes are taken into account. Almost all of the studies in this area focus on supervised techniques for spotting outliers. While supervised algorithms for anomaly detection work well in theory, they cannot be applied to real-world applications owing to a lack of labelled data. Considering the possible data distribution, our model employs a dual variational autoencoder (VAE), while a generative adversarial network (GAN) assures that the model is robust to adversarial training. The dual VAEs are used in another capacity: as a fake-node generator. Adversarial training is used to ensure that our latent codes have a Gaussian or uniform distribution. To provide a fair presentation of the graph, the discriminator instructs the generator to generate latent variables with distributions that are more consistent with the actual distribution of the data. Once the model has been learned, the discriminator is used for anomaly detection via reconstruction loss which has been trained to distinguish between the normal and artificial distributions of data. First, using a dual VAE, our model simultaneously captures cross-modality interactions between topological structure and node characteristics and overcomes the problem of unlabeled anomalies, allowing us to better understand the network sparsity and nonlinearity. Second, the proposed model considers the regularization of the latent codes while solving the issue of unregularized embedding techniques that can quickly lead to unsatisfactory representation. Finally, we use the discriminator reconstruction loss for anomaly detection as the discriminator is well-trained to separate the normal and generated data distributions because reconstruction-based loss does not include the adversarial component. Experiments conducted on attributed networks demonstrate the effectiveness of the proposed model and show that it greatly surpasses the previous methods. The area under the curve scores of our proposed model for the BlogCatalog, Flickr, and Enron datasets are 0.83680, 0.82020, and 0.71180, respectively, proving the effectiveness of the proposed model. The result of the proposed model on the Enron dataset is slightly worse than other models;we attribute this to the dataset’s low dimensionality as the most probable explanation.

Masked Autoencoders as Single Object Tracking Learners

Significant advancements have beenwitnessed in visual tracking applications leveragingViT in recent years,mainly due to the formidablemodeling capabilities of Vision Transformer(ViT).However,the strong performance of such trackers heavily relies on ViT models pretrained for long periods,limitingmore flexible model designs for tracking tasks.To address this issue,we propose an efficient unsupervised ViT pretraining method for the tracking task based on masked autoencoders,called TrackMAE.During pretraining,we employ two shared-parameter ViTs,serving as the appearance encoder and motion encoder,respectively.The appearance encoder encodes randomly masked image data,while the motion encoder encodes randomly masked pairs of video frames.Subsequently,an appearance decoder and a motion decoder separately reconstruct the original image data and video frame data at the pixel level.In this way,ViT learns to understand both the appearance of images and the motion between video frames simultaneously.Experimental results demonstrate that ViT-Base and ViT-Large models,pretrained with TrackMAE and combined with a simple tracking head,achieve state-of-the-art(SOTA)performance without additional design.Moreover,compared to the currently popular MAE pretraining methods,TrackMAE consumes only 1/5 of the training time,which will facilitate the customization of diverse models for tracking.For instance,we additionally customize a lightweight ViT-XS,which achieves SOTA efficient tracking performance.

Trusted Encrypted Traffic Intrusion Detection Method Based on Federated Learning and Autoencoder

With the rapid development of the Internet,network security and data privacy are increasingly valued.Although classical Network Intrusion Detection System(NIDS)based on Deep Learning(DL)models can provide good detection accuracy,but collecting samples for centralized training brings the huge risk of data privacy leakage.Furthermore,the training of supervised deep learning models requires a large number of labeled samples,which is usually cumbersome.The“black-box”problem also makes the DL models of NIDS untrustworthy.In this paper,we propose a trusted Federated Learning(FL)Traffic IDS method called FL-TIDS to address the above-mentioned problems.In FL-TIDS,we design an unsupervised intrusion detection model based on autoencoders that alleviates the reliance on marked samples.At the same time,we use FL for model training to protect data privacy.In addition,we design an improved SHAP interpretable method based on chi-square test to perform interpretable analysis of the trained model.We conducted several experiments to evaluate the proposed FL-TIDS.We first determine experimentally the structure and the number of neurons of the unsupervised AE model.Secondly,we evaluated the proposed method using the UNSW-NB15 and CICIDS2017 datasets.The exper-imental results show that the unsupervised AE model has better performance than the other 7 intrusion detection models in terms of precision,recall and f1-score.Then,federated learning is used to train the intrusion detection model.The experimental results indicate that the model is more accurate than the local learning model.Finally,we use an improved SHAP explainability method based on Chi-square test to analyze the explainability.The analysis results show that the identification characteristics of the model are consistent with the attack characteristics,and the model is reliable.

Abnormal State Detection in Lithium-ion Battery Using Dynamic Frequency Memory and Correlation Attention LSTM Autoencoder

This paper addresses the challenge of identifying abnormal states in Lithium-ion Battery(LiB)time series data.As the energy sector increasingly focuses on integrating distributed energy resources,Virtual Power Plants(VPP)have become a vital new framework for energy management.LiBs are key in this context,owing to their high-efficiency energy storage capabilities essential for VPP operations.However,LiBs are prone to various abnormal states like overcharging,over-discharging,and internal short circuits,which impede power transmission efficiency.Traditional methods for detecting such abnormalities in LiB are too broad and lack precision for the dynamic and irregular nature of LiB data.In response,we introduce an innovative method:a Long Short-Term Memory(LSTM)autoencoder based on Dynamic Frequency Memory and Correlation Attention(DFMCA-LSTM-AE).This unsupervised,end-to-end approach is specifically designed for dynamically monitoring abnormal states in LiB data.The method starts with a Dynamic Frequency Fourier Transform module,which dynamically captures the frequency characteristics of time series data across three scales,incorporating a memory mechanism to reduce overgeneralization of abnormal frequencies.This is followed by integrating LSTM into both the encoder and decoder,enabling the model to effectively encode and decode the temporal relationships in the time series.Empirical tests on a real-world LiB dataset demonstrate that DFMCA-LSTM-AE outperforms existing models,achieving an average Area Under the Curve(AUC)of 90.73%and an F1 score of 83.83%.These results mark significant improvements over existing models,ranging from 2.4%–45.3%for AUC and 1.6%–28.9%for F1 score,showcasing the model’s enhanced accuracy and reliability in detecting abnormal states in LiB data.

A strategy for out-of-roundness damage wheels identification in railway vehicles based on sparse autoencoders

Wayside monitoring is a promising cost-effective alternative to predict damage in the rolling stock. The main goal of this work is to present an unsupervised methodology to identify out-of-roundness(OOR) damage wheels, such as wheel flats and polygonal wheels. This automatic damage identification algorithm is based on the vertical acceleration evaluated on the rails using a virtual wayside monitoring system and involves the application of a two-step procedure. The first step aims to define a confidence boundary by using(healthy) measurements evaluated on the rail constituting a baseline. The second step of the procedure involves classifying damage of predefined scenarios with different levels of severities. The proposed procedure is based on a machine learning methodology and includes the following stages:(1) data collection,(2) damage-sensitive feature extraction from the acquired responses using a neural network model, i.e., the sparse autoencoder(SAE),(3) data fusion based on the Mahalanobis distance, and(4) unsupervised feature classification by implementing outlier and cluster analysis. This procedure considers baseline responses at different speeds and rail irregularities to train the SAE model. Then, the trained SAE is capable to reconstruct test responses(not trained) allowing to compute the accumulative difference between original and reconstructed signals. The results prove the efficiency of the proposed approach in identifying the two most common types of OOR in railway wheels.

Advancing Autoencoder Architectures for Enhanced Anomaly Detection in Multivariate Industrial Time Series

In the context of rapid digitization in industrial environments,how effective are advanced unsupervised learning models,particularly hybrid autoencoder models,at detecting anomalies in industrial control system(ICS)datasets?This study is crucial because it addresses the challenge of identifying rare and complex anomalous patterns in the vast amounts of time series data generated by Internet of Things(IoT)devices,which can significantly improve the reliability and safety of these systems.In this paper,we propose a hybrid autoencoder model,called ConvBiLSTMAE,which combines convolutional neural network(CNN)and bidirectional long short-term memory(BiLSTM)to more effectively train complex temporal data patterns in anomaly detection.On the hardware-in-the-loopbased extended industrial control system dataset,the ConvBiLSTM-AE model demonstrated remarkable anomaly detection performance,achieving F1 scores of 0.78 and 0.41 for the first and second datasets,respectively.The results suggest that hybrid autoencoder models are not only viable,but potentially superior alternatives for unsupervised anomaly detection in complex industrial systems,offering a promising approach to improving their reliability and safety.

A Trust Evaluation Mechanism Based on Autoencoder Clustering Algorithm for Edge Device Access of IoT

First,we propose a cross-domain authentication architecture based on trust evaluation mechanism,including registration,certificate issuance,and cross-domain authentication processes.A direct trust evaluation mechanism based on the time decay factor is proposed,taking into account the influence of historical interaction records.We weight the time attenuation factor to each historical interaction record for updating and got the new historical record data.We refer to the beta distribution to enhance the flexibility and adaptability of the direct trust assessment model to better capture time trends in the historical record.Then we propose an autoencoder-based trust clustering algorithm.We perform feature extraction based on autoencoders.Kullback leibler(KL)divergence is used to calculate the reconstruction error.When constructing a convolutional autoencoder,we introduce convolutional neural networks to improve training efficiency and introduce sparse constraints into the hidden layer of the autoencoder.The sparse penalty term in the loss function measures the difference through the KL divergence.Trust clustering is performed based on the density based spatial clustering of applications with noise(DBSCAN)clustering algorithm.During the clustering process,edge nodes have a variety of trustworthy attribute characteristics.We assign different attribute weights according to the relative importance of each attribute in the clustering process,and a larger weight means that the attribute occupies a greater weight in the calculation of distance.Finally,we introduced adaptive weights to calculate comprehensive trust evaluation.Simulation experiments prove that our trust evaluation mechanism has excellent reliability and accuracy.

Bridge damage identification based on convolutional autoencoders and extreme gradient boosting trees

To enhance the accuracy and efficiency of bridge damage identification,a novel data-driven damage identification method was proposed.First,convolutional autoencoder(CAE)was used to extract key features from the acceleration signal of the bridge structure through data reconstruction.The extreme gradient boosting tree(XGBoost)was then used to perform analysis on the feature data to achieve damage detection with high accuracy and high performance.The proposed method was applied in a numerical simulation study on a three-span continuous girder and further validated experimentally on a scaled model of a cable-stayed bridge.The numerical simulation results show that the identification errors remain within 2.9%for six single-damage cases and within 3.1%for four double-damage cases.The experimental validation results demonstrate that when the tension in a single cable of the cable-stayed bridge decreases by 20%,the method accurately identifies damage at different cable locations using only sensors installed on the main girder,achieving identification accuracies above 95.8%in all cases.The proposed method shows high identification accuracy and generalization ability across various damage scenarios.

Network Intrusion Detection Model Based on Ensemble of Denoising Adversarial Autoencoder

Network security problems bring many imperceptible threats to the integrity of data and the reliability of device services,so proposing a network intrusion detection model with high reliability is of great research significance for network security.Due to the strong generalization of invalid features during training process,it is more difficult for single autoencoder intrusion detection model to obtain effective results.A network intrusion detection model based on the Ensemble of Denoising Adversarial Autoencoder(EDAAE)was proposed,which had higher accuracy and reliability compared to the traditional anomaly detection model.Using the adversarial learning idea of Adversarial Autoencoder(AAE),the discriminator module was added to the original model,and the encoder part was used as the generator.The distribution of the hidden space of the data generated by the encoder matched with the distribution of the original data.The generalization of the model to the invalid features was also reduced to improve the detection accuracy.At the same time,the denoising autoencoder and integrated operation was introduced to prevent overfitting in the adversarial learning process.Experiments on the CICIDS2018 traffic dataset showed that the proposed intrusion detection model achieves an Accuracy of 95.23%,which out performs traditional self-encoders and other existing intrusion detection models methods in terms of overall performance.

Generalized autoencoder-based fault detection method for traction systems with performance degradation

Fault diagnosis of traction systems is important for the safety operation of high-speed trains.Long-term operation of the trains will degrade the performance of systems,which decreases the fault detection accuracy.To solve this problem,this paper proposes a fault detection method developed by a Generalized Autoencoder(GAE)for systems with performance degradation.The advantage of this method is that it can accurately detect faults when the traction system of high-speed trains is affected by performance degradation.Regardless of the probability distribution,it can handle any data,and the GAE has extremely high sensitivity in anomaly detection.Finally,the effectiveness of this method is verified through the Traction Drive Control System(TDCS)platform.At different performance degradation levels,our method’s experimental results are superior to traditional methods.

Network Intrusion Detection in Internet of Blended Environment Using Ensemble of Heterogeneous Autoencoders(E-HAE)

Contemporary attackers,mainly motivated by financial gain,consistently devise sophisticated penetration techniques to access important information or data.The growing use of Internet of Things(IoT)technology in the contemporary convergence environment to connect to corporate networks and cloud-based applications only worsens this situation,as it facilitates multiple new attack vectors to emerge effortlessly.As such,existing intrusion detection systems suffer from performance degradation mainly because of insufficient considerations and poorly modeled detection systems.To address this problem,we designed a blended threat detection approach,considering the possible impact and dimensionality of new attack surfaces due to the aforementioned convergence.We collectively refer to the convergence of different technology sectors as the internet of blended environment.The proposed approach encompasses an ensemble of heterogeneous probabilistic autoencoders that leverage the corresponding advantages of a convolutional variational autoencoder and long short-term memory variational autoencoder.An extensive experimental analysis conducted on the TON_IoT dataset demonstrated 96.02%detection accuracy.Furthermore,performance of the proposed approach was compared with various single model(autoencoder)-based network intrusion detection approaches:autoencoder,variational autoencoder,convolutional variational autoencoder,and long short-term memory variational autoencoder.The proposed model outperformed all compared models,demonstrating F1-score improvements of 4.99%,2.25%,1.92%,and 3.69%,respectively.

基于AutoEncoder的增量式聚类算法

针对目前数据量增长迅速,数据特征多,存储空间不足等问题,提出了基于AutoEncoder的增量式聚类算法(ANIC).首先利用AutoEncoder学习数据样本的特征,进行低维特征整合,得到数据样本的压缩表示形式,然后在原有聚类结果的基础上,通过一遍式读取数据和动态更新聚类中心点,对新生成样本进行增量式聚类.在对UCI的四个数据集进行聚类时,实验结果表明该算法能够得到优于k均值算法(Kmeans)和模糊c均值算法(FCM)的聚类效果.同时,该算法的时间消耗低,能够识别离群点,能够有效地对不断增加的数据集进行增量式聚类.

基于One Class-SVM+Autoencoder模型的车辆碰撞检测

为尽量避免车辆碰撞事故的发生,探索了机器学习和深度学习结合的方法,利用影响车辆碰撞的多个特征变量对车辆碰撞进行检测.首先使用皮尔逊相关性分析方法分析各个特征之间的关联度,接着使用One Class-SVM模型对数据集做"异常点"抛除操作.利用SMOTE(Synthetic Minority Over-sampling Technique)算法增加了少数类别的样本数量,最后采用自动编码器模型将影响车辆碰撞的因素(例如天气情况、光照情况等)作为模型的输入,通过解码器重构原始输入,获得输入与输出的最小重构误差计算阈值判断车辆碰撞情况.实验表明,数据经过One Class-SVM模型处理,再使用Autoencoder模型检测获得了比较好的测试结果.

Enhanced Deep Autoencoder Based Feature Representation Learning for Intelligent Intrusion Detection System

In the era of Big data,learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system(IDS).Owing to the lack of accurately labeled network traffic data,many unsupervised feature representation learning models have been proposed with state-of-theart performance.Yet,these models fail to consider the classification error while learning the feature representation.Intuitively,the learnt feature representation may degrade the performance of the classification task.For the first time in the field of intrusion detection,this paper proposes an unsupervised IDS model leveraging the benefits of deep autoencoder(DAE)for learning the robust feature representation and one-class support vector machine(OCSVM)for finding the more compact decision hyperplane for intrusion detection.Specially,the proposed model defines a new unified objective function to minimize the reconstruction and classification error simultaneously.This unique contribution not only enables the model to support joint learning for feature representation and classifier training but also guides to learn the robust feature representation which can improve the discrimination ability of the classifier for intrusion detection.Three set of evaluation experiments are conducted to demonstrate the potential of the proposed model.First,the ablation evaluation on benchmark dataset,NSL-KDD validates the design decision of the proposed model.Next,the performance evaluation on recent intrusion dataset,UNSW-NB15 signifies the stable performance of the proposed model.Finally,the comparative evaluation verifies the efficacy of the proposed model against recently published state-of-the-art methods.

Bi-LSTM-Based Deep Stacked Sequence-to-Sequence Autoencoder for Forecasting Solar Irradiation and Wind Speed

Wind and solar energy are two popular forms of renewable energy used in microgrids and facilitating the transition towards net-zero carbon emissions by 2050.However,they are exceedingly unpredictable since they rely highly on weather and atmospheric conditions.In microgrids,smart energy management systems,such as integrated demand response programs,are permanently established on a step-ahead basis,which means that accu-rate forecasting of wind speed and solar irradiance intervals is becoming increasingly crucial to the optimal operation and planning of microgrids.With this in mind,a novel“bidirectional long short-term memory network”(Bi-LSTM)-based,deep stacked,sequence-to-sequence autoencoder(S2SAE)forecasting model for predicting short-term solar irradiation and wind speed was developed and evaluated in MATLAB.To create a deep stacked S2SAE prediction model,a deep Bi-LSTM-based encoder and decoder are stacked on top of one another to reduce the dimension of the input sequence,extract its features,and then reconstruct it to produce the forecasts.Hyperparameters of the proposed deep stacked S2SAE forecasting model were optimized using the Bayesian optimization algorithm.Moreover,the forecasting performance of the proposed Bi-LSTM-based deep stacked S2SAE model was compared to three other deep,and shallow stacked S2SAEs,i.e.,the LSTM-based deep stacked S2SAE model,gated recurrent unit-based deep stacked S2SAE model,and Bi-LSTM-based shallow stacked S2SAE model.All these models were also optimized and modeled in MATLAB.The results simulated based on actual data confirmed that the proposed model outperformed the alternatives by achieving an accuracy of up to 99.7%,which evidenced the high reliability of the proposed forecasting.

Unsupervised Electric Motor Fault Detection by Using Deep Autoencoders

Fault diagnosis of electric motors is a fundamental task for production line testing, and it is usually performed by experienced human operators. In the recent years, several methods have been proposed in the literature for detecting faults automatically. Deep neural networks have been successfully employed for this task, but, up to the authors' knowledge, they have never been used in an unsupervised scenario. This paper proposes an unsupervised method for diagnosing faults of electric motors by using a novelty detection approach based on deep autoencoders. In the proposed method, vibration signals are acquired by using accelerometers and processed to extract LogMel coefficients as features. Autoencoders are trained by using normal data only, i.e., data that do not contain faults. Three different autoencoders architectures have been evaluated: the multilayer perceptron(MLP) autoencoder, the convolutional neural network autoencoder, and the recurrent autoencoder composed of long short-term memory(LSTM) units. The experiments have been conducted by using a dataset created by the authors, and the proposed approaches have been compared to the one-class support vector machine(OC-SVM) algorithm. The performance has been evaluated in terms area under curve(AUC) of the receiver operating characteristic curve, and the results showed that all the autoencoder-based approaches outperform the OCSVM algorithm. Moreover, the MLP autoencoder is the most performing architecture, achieving an AUC equal to 99.11 %.

融合Autoencoder方法的电力系统网络安全风险评估技术

针对现有电力系统安全评估指标存在高复杂度与低准确度等缺点,基于Autoencoder方法提出了适用于电力系统的网络安全风险评估方法.通过分析系统运行机制的脆弱性,使用层次分析法与专家调查法,建立了网络安全的初步评估指标体系模型.通过引入Autoencoder方法对复杂的指标体系模型进行必要的约简与优化,形成电力系统的新型安全评估模型.仿真结果表明,与传统的安全评估模型相比,所提出模型具有更高的执行效率与评估准确度.

Multi-Modal Domain Adaptation Variational Autoencoder for EEG-Based Emotion Recognition

Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.

Efficient Deep-Learning-Based Autoencoder Denoising Approach for Medical Image Diagnosis

Effective medical diagnosis is dramatically expensive,especially in third-world countries.One of the common diseases is pneumonia,and because of the remarkable similarity between its types and the limited number of medical images for recent diseases related to pneumonia,themedical diagnosis of these diseases is a significant challenge.Hence,transfer learning represents a promising solution in transferring knowledge from generic tasks to specific tasks.Unfortunately,experimentation and utilization of different models of transfer learning do not achieve satisfactory results.In this study,we suggest the implementation of an automatic detectionmodel,namelyCADTra,to efficiently diagnose pneumonia-related diseases.This model is based on classification,denoising autoencoder,and transfer learning.Firstly,pre-processing is employed to prepare the medical images.It depends on an autoencoder denoising(AD)algorithm with a modified loss function depending on a Gaussian distribution for decoder output to maximize the chances for recovering inputs and clearly demonstrate their features,in order to improve the diagnosis process.Then,classification is performed using a transfer learning model and a four-layer convolution neural network(FCNN)to detect pneumonia.The proposed model supports binary classification of chest computed tomography(CT)images and multi-class classification of chest X-ray images.Finally,a comparative study is introduced for the classification performance with and without the denoising process.The proposed model achieves precisions of 98%and 99%for binary classification and multi-class classification,respectively,with the different ratios for training and testing.To demonstrate the efficiency and superiority of the proposed CADTra model,it is compared with some recent state-of-the-art CNN models.The achieved outcomes prove that the suggested model can help radiologists to detect pneumonia-related diseases and improve the diagnostic efficiency compared to the existing diagnosis models.

Improved Denoising Autoencoder for Maritime Image Denoising and Semantic Segmentation of USV

Unmanned surface vehicle(USV)is currently a hot research topic in maritime communication network(MCN),where denoising and semantic segmentation of maritime images taken by USV have been rarely studied.The former has recently researched on autoencoder model used for image denoising,but the existed models are too complicated to be suitable for real-time detection of USV.In this paper,we proposed a lightweight autoencoder combined with inception module for maritime image denoising in different noisy environments and explore the effect of different inception modules on the denoising performance.Furthermore,we completed the semantic segmentation task for maritime images taken by USV utilizing the pretrained U-Net model with tuning,and compared them with original U-Net model based on different backbone.Subsequently,we compared the semantic segmentation of noised and denoised maritime images respectively to explore the effect of image noise on semantic segmentation performance.Case studies are provided to prove the feasibility of our proposed denoising and segmentation method.Finally,a simple integrated communication system combining image denoising and segmentation for USV is shown.