摘要
网络日志分析对网络安全管理具有重要意义,但现有的网络日志分析系统具有无法处理海量日志数据、采用离线模式、处理时延较长等弊端。因此,提出网络日志数据的分级归档机制,实现海量数据的分级优化存储功能。通过将传统的网络日志分析模式和基于Hadoop的离线处理模式相结合,设计基于Hadoop的在线网络日志分析系统,并深入分析系统的功能架构和业务处理流程。实验结果表明:该系统具有良好的可扩展性,可以有效存储处理海量日志数据,具有较快的请求响应速度,并能够满足在线处理的要求。从而验证了系统设计的可行性和有效性。
Network log analysis has vital significance to network security management,however existing network log analysis system has some drawbacks,such as unable to deal with huge amounts of log data,using offline mode,and longer processing delay,etc. Therefore,we propose the hierarchical archiving mechanism of network log data and realise the function of hierarchical and optimised storage of large amount of log data. Through the combination of traditional log analysis mode and Hadoop-based offline analysis mode,we design the Hadoop-based online log analysis system. Moreover,we analyse in depth the functional architecture and the transactions processing flow of the system.Experimental results show that the new system has good scalability and can effectively store and process massive log data. Furthermore,it has faster request and response speed which can meet the requirements of online processing. The feasibility and effectiveness of the system design is also verified.
出处
《计算机应用与软件》
CSCD
北大核心
2014年第8期311-316,共6页
Computer Applications and Software
基金
河南省科技攻关计划项目(122102310474)
