基于海量日志的入侵检测并行化算法研究

Research on intrusion detection parallel algorithm based on massive logs

随着计算机技术和互联网的迅猛发展,对海量日志进行分析并进行入侵检测就成为重要的研究问题。针对这一现象,提出在Hadoop平台下利用并行化的数据挖掘算法对海量的日志信息进行分析从而进行入侵检测,然后利用搭建好的Hadoop集群环境对其进行验证,对不同大小的日志文件进行处理,并与单机环境下对比,证明在该平台下进行入侵检测的有效性和高效性,同时实验证明如果增大集群中的节点数目,执行效率也会相应的提高。

With the rapid development of computer technology and Internet, how to analyze the massive logs and perform the intrusion detection become the important research contents. To soleve these difficulties, the parallel data mining algorithm is used to analyze the massive logs information on Hadoop platform, so as to perform the intrusion detection. The established Hadoop cluster environment is used to verify the intrusion detection, and process the log files with different sizes. In comparison with the intrusion detection result verified in the stand-alone environment, the effectiveness and efficiency of the intrusion detec- tion on Hadoop platform were verified. And the experiment results verify that if the node quantity in the cluster is increased, the execution efficiency will be improved accordingly.