基于双向长短时记忆网络的系统异常检测方法

SYSTEM ANOMALY DETECTION METHOD BASED ON BIDIRECTIONAL LSTM

在系统日志异常检测中,日志结构不统一且新执行的日志路径检测依然不够准确。针对这些问题,提出一种基于双向长短时记忆网络的日志路径异常检测模型。通过日志解析器构造日志键使得日志结构统一化,同时将日志键转化为时序序列构建时序化的日志结构;采用双向长短时记忆网络对时序化的日志序列进行建模和预测,根据是否发生误判来优化模型参数,提升新执行的日志路径检测效率。实验结果表明,与传统的基于机器学习的日志路径异常检测模型相比,该模型在HDFS和OpenStack数据集上准确率分别提升11%和20%,验证了该模型的有效性。

Inconsistency of log structure and the failure to detect new log paths accurately are main challenges of log anomaly detection.To address these challenges,a novel anomaly detection model of system log paths based on bidirectional LSTM is proposed.The log keys were constructed by using log parser to unify log structure,and log keys were converted into time series;a bidirectional LSTM was used to model and predict the sequential log sequence,and the model parameters were optimized according to whether misjudgement occurs,so as to improve the detection efficiency of the new execution log path.The experimental results show that compared with the traditional machine learning-based log path anomaly detection model,the accuracy of the model is improved by 11%and 20%respectively on HDFS and OpenStack datasets,which verifies the validity of the model.