基于等价类划分的并行频繁闭项集挖掘算法

Parallel Frequent Closed Itemsets Mining Algorithm Based on Equivalence Class Partition

针对海量数据的关联规则挖掘问题,提出了一种有效的基于等价类划分的并行频繁闭项集挖掘算法.该算法在MapReduce框架下,通过等价类的产生与划分、数据集的分配、异步频繁闭项集挖掘和汇总等步骤,不但较好地解决了多节点间的负载均衡问题,而且易于获得可靠的频繁闭项集.实验表明,该算法能有效克服传统算法挖掘效率低、冗余规则较多的缺点,整体上具有较高的性能.

For the problems of association rules mining of massive database,an effective parallel approach for the closed frequent itemsets mining based on the division of equivalence classes was presented. Under the framework of MapReduce,the proposed approach performs through three steps： 1） the division of equivalence class,2） the allocation of data set,and 3） the asynchronous mining and aggregation of frequent closed itemsets. Such a strategy can significantly solve the load balancing problem of multiple nodes and obtain the reliable frequent closed itemsets. Experimental results showed that the approach can effectively overcome the drawbacks of traditional approaches such as low efficiency of mining,more redundant rules and so on,and gain higher performance.