摘要
将传统的关联规则Apriori算法部署到云平台,数据挖掘的效率有很大的提升,但数据类型复杂程度和数据量的不断增加,Apriori算法需频繁扫描事务数据库,产生的频繁项明显增多,耗费大量的系统I/O、内存资源。提出一种基于Map/Reduce编程模型的并行HD-Apriori算法。该算法通过采用0-1矩阵从而达到简化计算并有效减少扫描数据库次数,实现Apriori算法的并行化。实验证明,HD-Apriori算法的挖掘效率优于Apriori算法,且数据规模越大,效果愈显著。
The traditional association rules Apriori algorithm is deployed to the cloud platform,the efficiency of data mining has a lot of improvement,but the data types complexity and increasing the amount of data,Apriori algorithm needs to scan the transaction database frequently,the frequent items significantly increased, large quantities of system I/O, memory resources. A parallel HD-Apriori algorithm based on the Map/Reduce programming model is proposed. The algorithm achieves the parallelization of Apriori algorithm by adopting the 0-1 matrix to simplify the computation and reduce the number of scanning database. The results show that the efficiency of the HD-Apriori algorithm is better than the Apriori algorithm,and the larger the data scale is,the more effective it is.
作者
石波
万定生
姚建国
丁伟
SHI Bo;WAN Ding-sheng;YAO Jian-guo;DING Wei(School of Computer and Information,Hohai University,Nanjing 210098,China;Bureau of Hydrology,Huaihe River Commission,Bengbu 233001,Anhui Province,China)
出处
《信息技术》
2018年第6期7-11,16,共6页
Information Technology
基金
国家科技支撑计划课题(2015BAB07B01)
公益性行业科研专项(201501022)
