基于随机Transformer的多维时间序列异常检测模型

Anomaly detection model for multivariate time series based on stochastic Transformer

针对已有基于变分自编码器(VAE)的多维时间序列(MTS)异常检测模型无法在隐空间中传播随机变量间的长时依赖性问题,提出了一种融合Transformer编码器和VAE的随机Transformer MTS异常检测模型(ST-MTS-AD)。在ST-MTS-AD的推断网络中,Transformer编码器产生的当前时刻MTS长时依赖特征和上一时刻随机变量的采样值被输入多层感知器,由此生成当前时刻随机变量的近似后验分布,实现随机变量间的时序依赖。采用门控转换函数(GTF)生成随机变量的先验分布,ST-MTS-AD的生成网络由多层感知器重构MTS各时刻取值分布,该多层感知器的输入为推断网络生成的MTS的长时依赖特征和随机变量近似后验采样值。ST-MTS-AD基于变分推断技术学习正常MTS样本集分布,由重构概率对数似然确定MTS异常片段。4个公开数据集上的实验表明,ST-MTS-AD模型比典型相关基线模型的F1分数有明显提升。

Aiming at the problem that the existing multivariate time series anomaly detection models based on variational autoencoders could not propagate long-term temporal dependencies between stochastic variables in latent space,the stochastic Transformer for MTS anomaly detection(ST-MTS-AD)model which combined Transformer encoder with VAE was proposed.In the inference network of the ST-MTS-AD,the MTS long-term temporal dependent features generated by Transformer encoder and the sampled values of the stochastic variables at the previous moment were inputted into the multilayer perceptron,the approximate posterior distribution of the stochastic variables at the current moment was generated by the multilayer perceptron,and the temporal dependencies between stochastic variables were realized.The gated transition function(GTF)was used to generate the prior distribution of stochastic variables.The generation network of the ST-MTS-AD reconstructed the distribution of the MTS values at each moment by the multilayer perceptron whose input was the MTS long-term temporal dependent features generated by the inference network and the approximate posterior sampling values of stochastic variables.The distribution of normal MTS dataset was learned by the variational inference technology,and the abnormal MTS segment was determined by the log-likelihood of the reconstruction probability.Experiments on four public datasets show that the ST-MTS-AD model significantly improves the F1 score over the typical baseline models.