基于时序数据库的用户访问日志分析系统设计

Design of user access log analysis system based on time series database

针对电力网络复杂程度日益增高,电力设备种类繁杂、数量日趋庞大,电力系统面临大规模信息化数据运维管理的问题。文中基于Spark大数据并行计算框架,设计了一种针对电力系统运维日志的数据分析系统,以实现分类处理海量数据。该系统利用服务器网络接口实现旁路监听的数据采集功能,在传统数据预处理算法的基础上采用了改进后的“两段式”数据预处理方法,使海量数据的分配过程更加高效、准确。同时,基于时序数据库和神经网络算法提出了时间序列系统异常预测算法。通过对时序数据库中的日志数据处理和分析,从而实现系统故障的监测与预警。经测试,该系统可有效监测电力系统运行状况,实现故障监测和预警,提高系统运行的安全性。

In view of the increasing complexity of power network and the increasing variety and quantity of power equipment,power system is facing the problem of large-scale information data operation and maintenance management.Based on Spark parallel computing framework for large data,a data analysis system for power system operation and maintenance log is designed to classify and process massive data.The system uses the server network interface to realize the data acquisition function of bypass monitoring.On the basis of the traditional data preprocessing algorithm,the improved"two-stage"data preprocessing method is adopted to make the distribution process of massive data more efficient and accurate.At the same time,an anomaly prediction algorithm for time series system is proposed based on time series database and neural network algorithm.By processing and analyzing log data in time series database,system fault monitoring and early warning can be realized.After testing,the system can effectively monitor the operation status of power system,realize fault monitoring and early warning,and improve the security of system operation.