全局通讯网络模式的数据挖掘方法研究

Parallel Algorithm for Mining Association Rules Based on Master-Worker Model

随着信息技术迅速发展,数据库的规模不断扩大,从而产生了大量的数据。如果使用传统的数据挖掘技术从庞大的数据中挖掘出有价值的规则、模式等将需要巨大的计算资源,而且会花费很长的时间。在考虑到挖掘的效率,负载平衡,运行环境,节点状态等多方面因素的基础上,文章提出了新的并行数据挖掘算法。各个并行计算单元之间采用全局通讯模式——Master-Worker模式来进行互相通信,降低了并行数据挖掘的通信成本,提高了挖掘的效率,缩短了挖掘的时间。最后,实验结果验证了该算法的有效性以及在大数据集挖掘应用中的优越性。

With the rapid development of information technology, expanding the size of the database, results in a large amount of data. If we use traditional data mining techniques from a large data to get valuable rules, models and so on. It will require enormous computing resources, and will take a long time. Taking into account the efficiency of the mining, load balancing, the running environment, node status and other factors, the paper proposes a new parallel data mining algorithm, the various parallel computing units use Master-Worker model for communication with each other. The proposed algorithm reduces the communication costs of parallel data mining, improves the efficiency of data mining, and shortens the period time of data mining. The experiment results demonstrate its effectiveness and superiority of the algorithm in large size data mining apphcation.