一种基于位图计算并行挖掘大数据频繁模式算法

Parallel Mining Frequent Patterns in Big Data Based on Bitmap Computation

设计了一种基于MapReduce框架并行挖掘大数据频繁模式的算法,算法首先研究了运用位图计算发现数据集频繁模式的方法;并对传统MapReduce框架进行扩展,增加了位图计算和不重要模式剪枝等计算功能;为了提高大数据模式挖掘的性能,还设计模式剪枝算法来识别并删除数据集中的不重要模式.最后,实验结果表明,该算法具有很强的可扩展性,并优于其它同类算法.

This paper proposed a parallel algorithm of mining frequent patterns in big data using extended MapReduce. First,we analyze the frequent pattern mining method using bitmap computation by scanning the dataset only once. Secondly,we extended the traditional MapReduce frame by adding the bitmap computation and frequent mining function. In order to improve the performance of mining big data,an algorithm of pruning insignificant patterns in dataset is also presented. Finally,the experimental results show that the proposed method is efficient,strong in scalability,and prior to analogous algorithms.