基于MapReduce的Web日志挖掘算法研究

Research on Web Log Mining Algorithm Based on MapReduce

使用Apriori算法进行日志数据挖掘发现用户的频繁访问页面,对商业智能有重要意义。面对日益增长的日志信息,基于单一节点的Web数据挖掘系统存在计算瓶颈,针对该问题,利用MapReduce强大的计算能力设计一种基于MapReduce架构的Web日志挖掘算法,为进一步验证算法,在Hadoop平台上利用改进后的算法挖掘用户的频繁访问页面。实验结果表明,基于MapReduce架构的Apriori算法,可以明显提高Web数据挖掘的效率。

Uses Apriori algorithm in Web log mining to mine frequent page is important to business intelligence. When processing the ever-growing data, there exists a bottleneck in current data mining system based on single node. To solve these problems, proposes a Web log mining al- gorithm based on MapReduce by using the strong computing power of MapReduce.To proving the algorithm, uses improved algorithm to mining frequent pages on Hadoop. The experiments show that the algorithm on MapReduce can remarkably improve execution efficiency of the Web log mining.