基于Hadoop平台下日志分析系统的研究

Study on the log analysis system based on Hadoop platform

在信息系统当中,日志数据也就是用户对系统的操作记录以及系统自身的运行状况记录,对日志数据的有效分析,有利于排除系统故障、明确用户访问规律等。基于此,文章就Hadoop平台下日志分析系统展开研究,首先介绍了系统的总体框架,进一步对其设计与实现过程中,日志分析处理流程设计、日志数据预处理两个操作环节进行了重点介绍,并进一步提出了针对此类系统的检测方法。

In the information system, the log data is the user＇s operation record of the system and the system＇s own operation record. The effective analysis of the log data is helpful to rule out the system failure and to clarify the rules of user access. Based on this, the paper studies the log analysis system under the Hadoop platform, first introduces the overall framework of the system, and further introduces the two operation steps in the design and implementation process, log analysis and process flow design, log data preprocessing, and further proposed for the detection of such systems.