基于MapReduce的关联规则增量更新算法

MapReduce Based Association Rule Incremental Updating Algorithm

云计算以其强大的存储和计算能力而成为解决海量数据挖掘问题的有效途径。经典的关联规则增量更新算法FUP需要频繁扫描原数据集,不适用于海量数据的处理。文中以提高海量数据上关联规则增量更新效率为目标,将FUP算法与云计算的MapReduce编程模式相结合,提出了一种基于MapReduce的关联规则增量更新算法MRFUP。该算法只需扫描原数据集一次,并能充分利用云计算强大的存储和并行计算能力。基于Hadoop的实验结果表明,MRFUP算法可提高对海量数据的处理能力和效率,适用于海量数据的关联规则挖掘。

Cloud computing,with its powerful storage and computing power,has become one of the most effective way for solving the problem of massive data mining.FUP is one of the most classic incremental updating algorithms for association rules.But it can not meet the need of massive data mining very well because it needs to scan the dataset frequently.In this paper,in order to enhance the incremental updating efficiency of association rules for massive data,a MapReduce based incremental updating algorithm for association rules is proposed by combing FUP algorithm and MapReduce programming mode,which is named MRFUP.MRFUP scans the original dataset only once,and takes full advantage of the powerful storage and computing power provided by cloud computing.The results of the experiments deployed on Hadoop show that MRFUP can improve the ability and efficiency of processing massive data;It adapts to mine association rules from massive data.