Unsupervised Anomaly Detection Approach Based on Adversarial Memory Autoencoders for Multivariate Time Series

The widespread usage of Cyber Physical Systems(CPSs)generates a vast volume of time series data,and precisely determining anomalies in the data is critical for practical production.Autoencoder is the mainstream method for time series anomaly detection,and the anomaly is judged by reconstruction error.However,due to the strong generalization ability of neural networks,some abnormal samples close to normal samples may be judged as normal,which fails to detect the abnormality.In addition,the dataset rarely provides sufficient anomaly labels.This research proposes an unsupervised anomaly detection approach based on adversarial memory autoencoders for multivariate time series to solve the above problem.Firstly,an encoder encodes the input data into low-dimensional space to acquire a feature vector.Then,a memory module is used to learn the feature vector’s prototype patterns and update the feature vectors.The updating process allows partial forgetting of information to prevent model overgeneralization.After that,two decoders reconstruct the input data.Finally,this research uses the Peak Over Threshold(POT)method to calculate the threshold to determine anomalous samples from normal samples.This research uses a two-stage adversarial training strategy during model training to enlarge the gap between the reconstruction error of normal and abnormal samples.The proposed method achieves significant anomaly detection results on synthetic and real datasets from power systems,water treatment plants,and computer clusters.The F1 score reached an average of 0.9196 on the five datasets,which is 0.0769 higher than the best baseline method.

Network Intrusion Detection Model Based on Ensemble of Denoising Adversarial Autoencoder

Network security problems bring many imperceptible threats to the integrity of data and the reliability of device services,so proposing a network intrusion detection model with high reliability is of great research significance for network security.Due to the strong generalization of invalid features during training process,it is more difficult for single autoencoder intrusion detection model to obtain effective results.A network intrusion detection model based on the Ensemble of Denoising Adversarial Autoencoder(EDAAE)was proposed,which had higher accuracy and reliability compared to the traditional anomaly detection model.Using the adversarial learning idea of Adversarial Autoencoder(AAE),the discriminator module was added to the original model,and the encoder part was used as the generator.The distribution of the hidden space of the data generated by the encoder matched with the distribution of the original data.The generalization of the model to the invalid features was also reduced to improve the detection accuracy.At the same time,the denoising autoencoder and integrated operation was introduced to prevent overfitting in the adversarial learning process.Experiments on the CICIDS2018 traffic dataset showed that the proposed intrusion detection model achieves an Accuracy of 95.23%,which out performs traditional self-encoders and other existing intrusion detection models methods in terms of overall performance.

Anomalous node detection in attributed social networks using dual variational autoencoder with generative adversarial networks

Many types of real-world information systems, including social media and e-commerce platforms, can be modelled by means of attribute-rich, connected networks. The goal of anomaly detection in artificial intelligence is to identify illustrations that deviate significantly from the main distribution of data or that differ from known cases. Anomalous nodes in node-attributed networks can be identified with greater precision if both graph and node attributes are taken into account. Almost all of the studies in this area focus on supervised techniques for spotting outliers. While supervised algorithms for anomaly detection work well in theory, they cannot be applied to real-world applications owing to a lack of labelled data. Considering the possible data distribution, our model employs a dual variational autoencoder (VAE), while a generative adversarial network (GAN) assures that the model is robust to adversarial training. The dual VAEs are used in another capacity: as a fake-node generator. Adversarial training is used to ensure that our latent codes have a Gaussian or uniform distribution. To provide a fair presentation of the graph, the discriminator instructs the generator to generate latent variables with distributions that are more consistent with the actual distribution of the data. Once the model has been learned, the discriminator is used for anomaly detection via reconstruction loss which has been trained to distinguish between the normal and artificial distributions of data. First, using a dual VAE, our model simultaneously captures cross-modality interactions between topological structure and node characteristics and overcomes the problem of unlabeled anomalies, allowing us to better understand the network sparsity and nonlinearity. Second, the proposed model considers the regularization of the latent codes while solving the issue of unregularized embedding techniques that can quickly lead to unsatisfactory representation. Finally, we use the discriminator reconstruction loss for anomaly detection as the discriminator is well-trained to separate the normal and generated data distributions because reconstruction-based loss does not include the adversarial component. Experiments conducted on attributed networks demonstrate the effectiveness of the proposed model and show that it greatly surpasses the previous methods. The area under the curve scores of our proposed model for the BlogCatalog, Flickr, and Enron datasets are 0.83680, 0.82020, and 0.71180, respectively, proving the effectiveness of the proposed model. The result of the proposed model on the Enron dataset is slightly worse than other models;we attribute this to the dataset’s low dimensionality as the most probable explanation.

基于LAAE网络的跨语言短文本情感分析方法

跨语言短文本情感分析作为自然语言处理领域的一项重要的任务,近年来备受关注.跨语言情感分析能够利用资源丰富的源语言标注数据对资源匮乏的目标语言数据进行情感分析,建立语言之间的联系是该任务的核心.与传统的机器翻译建立联系方法相比,迁移学习更胜一筹,而高质量的跨语言文本向量则会提升迁移效果.本文提出LAAE网络模型,该模型通过长短记忆网络(LSTM)和对抗式自编码器(AAE)获得含上下文情感信息的跨语言向量,然后利用双向GRU (Gated Recurrent Unite)进行后续情感分类任务.其中,分类器首先在源语言上进行训练,最后迁移到目标语言上进行分类任务.本方法的有效性体现在实验结果中.

Primary User Adversarial Attacks on Deep Learning-Based Spectrum Sensing and the Defense Method

The spectrum sensing model based on deep learning has achieved satisfying detection per-formence,but its robustness has not been verified.In this paper,we propose primary user adversarial attack(PUAA)to verify the robustness of the deep learning based spectrum sensing model.PUAA adds a care-fully manufactured perturbation to the benign primary user signal,which greatly reduces the probability of detection of the spectrum sensing model.We design three PUAA methods in black box scenario.In or-der to defend against PUAA,we propose a defense method based on autoencoder named DeepFilter.We apply the long short-term memory network and the convolutional neural network together to DeepFilter,so that it can extract the temporal and local features of the input signal at the same time to achieve effective defense.Extensive experiments are conducted to eval-uate the attack effect of the designed PUAA method and the defense effect of DeepFilter.Results show that the three PUAA methods designed can greatly reduce the probability of detection of the deep learning-based spectrum sensing model.In addition,the experimen-tal results of the defense effect of DeepFilter show that DeepFilter can effectively defend against PUAA with-out affecting the detection performance of the model.

一种基于AAE的协同多播主动缓存方案

随着用户终端数量的激增和5G技术的发展,形成了宏基站和小基站并存的网络。同时超高清视频、云VR/AR等应用对时延提出了更高的要求。为了缩短5G网络中的时延,文中结合小基站协同、多播和用户行为可预测的特性,提出了一种基于对抗自动编码(Adversarial Autoencoders, AAE)的协同多播主动缓存方案(Collaborative Multicast Proactive Caching Scheme Based on Adversarial Autoencoders, CMPCAAE)。该方案首先根据用户的特征信息将用户划分成偏好不同的用户组,然后通过AAE预测每个用户组可能请求的内容。为了减少缓存内容的冗余,采用蚁群算法(Ant Colony, ACO)将预测的内容预先部署到各小基站以实现小基站间的协同。在内容分发阶段,若分组中用户请求的是流行度高的内容,则以多播的方式将该内容主动缓存到分组中其他未发送请求的用户,否则以正常的方式进行分发。仿真结果表明,CMPCAAE方案在系统的平均请求时延和丢失率方面均优于经典的缓存方案。

Fooling intrusion detection systems using adversarially autoencoder

Due to the increasing cyber-attacks,various Intrusion Detection Systems(IDSs)have been proposed to identify network anomalies.Most existing machine learning-based IDSs learn patterns from the features extracted from network traffic flows,and the deep learning-based approaches can learn data distribution features from the raw data to differentiate normal and anomalous network flows.Although having been used in the real world widely,the above methods are vulnerable to some types of attacks.In this paper,we propose a novel attack framework,Anti-Intrusion Detection AutoEncoder(AIDAE),to generate features to disable the IDS.In the proposed framework,an encoder transforms features into a latent space,and multiple decoders reconstruct the continuous and discrete features,respectively.Additionally,a generative adversarial network is used to learn the flexible prior distribution of the latent space.The correlation between continuous and discrete features can be kept by using the proposed training scheme.Experiments conducted on NSL-KDD,UNSW-NB15,and CICIDS2017 datasets show that the generated features indeed degrade the detection performance of existing IDSs dramatically.

Missing Value Imputation Model Based on Adversarial Autoencoder Using Spatiotemporal Feature Extraction

Recently,the importance of data analysis has increased significantly due to the rapid data increase.In particular,vehicle communication data,considered a significant challenge in Intelligent Transportation Systems(ITS),has spatiotemporal characteristics and many missing values.High missing values in data lead to the decreased predictive performance of models.Existing missing value imputation models ignore the topology of transportation net-works due to the structural connection of road networks,although physical distances are close in spatiotemporal image data.Additionally,the learning process of missing value imputation models requires complete data,but there are limitations in securing complete vehicle communication data.This study proposes a missing value imputation model based on adversarial autoencoder using spatiotemporal feature extraction to address these issues.The proposed method replaces missing values by reflecting spatiotemporal characteristics of transportation data using temporal convolution and spatial convolution.Experimental results show that the proposed model has the lowest error rate of 5.92%,demonstrating excellent predictive accuracy.Through this,it is possible to solve the data sparsity problem and improve traffic safety by showing superior predictive performance.

Deep convolutional adversarial graph autoencoder using positive pointwise mutual information for graph embedding

Graph embedding aims to map the high-dimensional nodes to a low-dimensional space and learns the graph relationship from its latent representations.Most existing graph embedding methods focus on the topological structure of graph data,but ignore the semantic information of graph data,which results in the unsatisfied performance in practical applications.To overcome the problem,this paper proposes a novel deep convolutional adversarial graph autoencoder(GAE)model.To embed the semantic information between nodes in the graph data,the random walk strategy is first used to construct the positive pointwise mutual information(PPMI)matrix,then,graph convolutional net-work(GCN)is employed to encode the PPMI matrix and node content into the latent representation.Finally,the learned latent representation is used to reconstruct the topological structure of the graph data by decoder.Furthermore,the deep convolutional adversarial training algorithm is introduced to make the learned latent representation conform to the prior distribution better.The state-of-the-art experimental results on the graph data validate the effectiveness of the proposed model in the link prediction,node clustering and graph visualization tasks for three standard datasets,Cora,Citeseer and Pubmed.

基于对抗自编码填补网络的MIMO雷达故障阵元缺失数据重构

多输入多输出(Multiple In Multiple Out,MIMO)雷达的阵元故障会导致其协方差矩阵出现整行整列数据缺失,从而降低其角度估计性能。为此,提出一种对抗自编码填补网络(Adversarial Autoencoder Imputation Network,AAEIN)来重构故障阵元的缺失数据。该网络由负责重构缺失数据的自动编码(Autoencoder,AE)网络和负责分辨数据来源的鉴别器组成。在二者的对抗训练中,AE网络的重构能力和鉴别器的分辨能力不断得到提升,直至两者收敛。为避免网络训练过程中参数量大和计算复杂度高的问题,文中结合MIMO雷达协方差矩阵的Hermitian特性,使用协方差矩阵上三角部分构建数据集用于网络训练。仿真结果表明,文中方法可以有效地重构故障阵元的缺失数据且具有较高的重构精度。

面向不平衡图像数据的对抗自编码器过采样算法

许多适用于低维数据的传统不平衡学习算法在图像数据上的效果并不理想。基于生成对抗网络(GAN)的过采样算法虽然可以生成高质量图像,但在类不平衡情况下容易产生模式崩溃问题。基于自编码器(AE)的过采样算法容易训练,但生成的图像质量较低。为进一步提高过采样算法在不平衡图像中生成样本的质量和训练的稳定性,该文基于生成对抗网络和自编码器的思想提出一种融合自编码器和生成对抗网络的过采样算法(BAEGAN)。首先在自编码器中引入一个条件嵌入层,使用预训练的条件自编码器初始化GAN以稳定模型训练;然后改进判别器的输出结构,引入一种融合焦点损失和梯度惩罚的损失函数以减轻类不平衡的影响;最后从潜在向量的分布映射中使用合成少数类过采样技术(SMOTE)来生成高质量的图像。在4个图像数据集上的实验结果表明该算法在生成图像质量和过采样后的分类性能上优于具有辅助分类器的条件生成对抗网络(ACGAN)、平衡生成对抗网络(BAGAN)等过采样算法,能有效解决图像数据中的类不平衡问题。

基于cGAN-SAE的室内定位指纹生成方法

针对室内定位中指纹采集成本高、构建数据集难等问题,提出了一种基于条件稀疏自编码生成对抗网络的室内定位指纹生成方法。该方法通过增加自编码器隐藏层和输出层,增强了特征提取能力,引导生成器学习并生成指纹数据的关键特征。利用指纹选择算法筛选出最相关的指纹数据,扩充至指纹数据库中,并用于训练卷积长短时记忆网络模型以进行在线效果评估。实验结果表明,条件稀疏自编码生成对抗网络在不增加采集样本的情况下,提高了多栋多层建筑室内定位的精度。与原始条件生成对抗网络模型相比,在UJIIndoorLoc数据集上的预测中,定位误差降低了6%;在实际应用中,定位误差降低了14%。

基于对抗样本和自编码器的鲁棒异常检测

基于自编码器的异常检测方法仅利用正常样本进行训练,因此可以有效地重构正常样本,但不能较好地对异常样本进行重构。另外,当基于自编码器的异常检测方法受到对抗攻击时,往往会取得错误的检测结果。为了解决上述问题,提出了一种基于对抗样本和自编码器的鲁棒异常检测(Robust Anomaly Detection Based on Adversarial Samples and AutoEncoder,RAD-ASAE)方法。RAD-ASAE由两个参数共享的编码器和一个解码器构成。首先,对正常样本施加微小的扰动以生成对抗样本,利用正常样本和对抗样本同时对模型进行训练,以提高模型的对抗鲁棒性;其次,在样本空间中最小化对抗样本的重构误差以及正常样本与对抗样本的重构样本之间的均方误差,同时在潜在空间中最小化正常样本和对抗样本的潜在特征之间的均方误差,以提高自编码器的重构能力。在MNIST,Fashion-MNIST,CIFAR-10数据集上进行实验,结果表明,与7种相关方法相比,RAD-ASAE展现了更优的异常检测性能。

基于对抗二次自编码器和集成学习的工业过程早期故障检测

由于工业过程早期微小故障存在数据振幅小,特征强耦合的特点,导致传统自编码器模型对此类故障的检测效果不佳,对此,提出一种基于对抗二次自编码器和集成学习的工业过程早期故障检测方法。首先引入一种二次型神经元嵌入普通自编码器模型的隐藏层中,增加模型的表达能力,其次提出一种对抗性的二次自编码器,在训练过程中引入GAN网络,使自编码器的特征学习遵循特定的概率分布。然后利用集成学习思想对正常工况数据进行采样,给每个采样的子集训练一个对抗二次自编码器,利用每个子模型的SPE和T 2统计量分别生成两个矩阵,接着在生成的矩阵上使用单步滑动窗口内奇异值分解的融合策略,将每个窗口内的最大奇异值作为检测统计量。使用一个数值例子和TE过程对所提方法进行验证。实验结果表明所提方法具有良好的早期微小故障检测性能。

一种结合卷积自编码和补丁惩罚的生成对抗网络单图像去雨方法

针对传统的图像去雨方法存在去雨图像失真、生成伪影等问题,提出一种结合卷积自编码和补丁惩罚的生成对抗网络单图像去雨方法。首先,该方法采用卷积自编码组成生成器网络,使用对称跳跃连接提高生成器网络的训练效率和收敛性能,实现对图像细节信息和二维信号空间信息的重构;其次,引入马尔可夫鉴别器在图像补丁层次上进行惩罚,去除生成图像中的伪影;最后,提出一种新的精细化损失函数参与训练网络模型,进一步增强模型的去雨深度。采用峰值信噪比和结构相似性作为模型的评价标准,实验结果表明,该方法在现实雨图和合成雨图的去雨处理上都有良好的表现,基本还原了图像细节内容,并保证了较高的视觉质量。

基于对抗修复网络的输送带表面缺陷检测

针对输送带缺陷数据获取和标注困难、输送带工作场景中的不稳定因素和数据波动导致基于深度学习的输送带缺陷检测方法精度低的问题,提出了一种基于对抗修复网络的输送带表面缺陷检测模型。该模型主要由自编码器结构的生成器和马尔可夫判别器组成。在训练阶段,将模拟的输送带表面缺陷图像输入生成器,得到无模拟缺陷的重构图像,提升模型对未知缺陷的泛化能力;将原始无损输送带图像、重构图像和模拟的输送带表面缺陷图像输入马尔可夫判别器,通过残差块获得特征图,提高模型对于微小缺陷的检测能力。在检测阶段,将待测图像输入训练完的生成器得到重构图像,再通过训练完的马尔可夫判别器提取待测图像与重构图像的特征图,根据待测图像与重构图像特征图之间的均方误差和待测图像特征图最大值,计算异常分数并与设定的阈值进行比较,从而判断待测图像是否存在缺陷。实验结果表明,该模型的接收操作特征曲线下面积(ROC-AUC)达0.999,精确率-召回率曲线下面积(PR-AUC)达0.997,单张图像检测时间为13.51 ms,能准确定位不同类型缺陷位置。

双自编码结合变分贝叶斯的单细胞RNA-Seq聚类

近年来单细胞RNA测序(scRNA-seq)技术的快速发展使得在单个细胞水平上研究组织器官的异质性成为可能。针对单细胞RNA测序数据中准确鉴定细胞类型问题,提出一种新的基于双自编码结合变分贝叶斯高斯混合模型的聚类方法,称之为sc-VBDAE。首先通过对抗自编码网络的编码和解码过程重构数据,然后使用经典自编码对数据进行降维,获得低维且有效的数据。最后使用变分贝叶斯高斯混合模型对细胞进行聚类,并可视化聚类结果。在10个scRNA-seq数据上的实验结果表明,该方法在6个数据集上ARI指标均优于其它方法,在数据集Biase和Klein上ARI指标值达到0.90及以上。

基于图像对抗卷积自编码的燃烧稳定性定量监测

燃烧稳定性的准确监测对于优化燃烧状态具有重要意义。传统燃烧稳定性监测方法不仅高度依赖先验专家知识,而且难以实现定量评估。为了克服这些局限性,该文提出一种新的燃烧稳定性定量评价方法。该方法建立一种对抗卷积自编码(adversarial convolutional autoencoder,ACAE)提取火焰图像的深层特征,并利用一种定量评价指标进行特征分析。其中,ACAE采用一种新型对抗机制来提高训练效率,从而增强特征学习能力;定量评价指标的数值区间为[0,1],且当评价指标低于0.5时,燃烧状态稳定。通过乙烯燃烧实验测试了稳定性监测方法的可行性。结果表明,由ACAE提取的深层图像特征能够用于定量估计燃烧稳定性。此外,该监测方法具有较强的泛化性能,能够准确识别训练数据集范围以外的火焰图像。

深层图注意力对抗变分自动编码器

现有的图自动编码器忽视了图邻居节点的差异和图潜在的数据分布。为了提高图自动编码器嵌入能力,提出图注意力对抗变分自动编码器(AAVGA-d),该方法将注意力引入编码器,并在嵌入训练中使用对抗机制。图注意力编码器实现了对邻居节点权重的自适应分配,对抗正则化使编码器生成的嵌入向量分布接近数据的真实分布。为了加深图注意力层数,设计一种针对注意力网络的随机边删除技术(RDEdge),减少了层数过深引起的过平滑信息丢失。实验结果表明,AAVGA-d的图嵌入能力与目前流行的图自动编码器相比具有竞争优势。

基于对抗性双通道编码器的网络入侵检测算法

针对网络流量数据不平衡引起少数类攻击检测率低的问题,提出一种基于对抗性双通道编码器的入侵检测算法。分别采用正常流量和攻击流量来训练变分自编码器模型,构建基于自编码器派生流量数据的多通道表示形式的新特征向量,驱动生成对抗网络的生成过程朝向目标类,生成的少数类图像,有效地扩充数据集;通过引入CBAM模块来改进生成器的网络结构,融合通道和空间两个方向的特征,增强模型的特征提取能力;将判别器输出调整为单目标分类并加入softmax层,输出Fake、Normal和Attack结果,避免生成器生成无法与所需类型匹配的图像而获得奖励,提高生成图片的质量。实验结果表明,该方法能够有效降低误报率以及提高未知攻击的检测精度,尤其在不平衡数据集中具有更多的优势。