面向海量数据的K-means聚类优化算法

Optimized K-means clustering algorithm for massive data

针对集中式系统框架难以进行海量数据聚类分析的问题,提出基于MapReduce的K-means聚类优化算法。该算法运用MapReduce并行编程框架,引入Canopy聚类,优化K-means算法初始中心的选取,改进迭代过程中通信和计算模式。实验结果表明该算法能够有效地改善聚类质量,具有较高的执行效率以及优良的扩展性,适合用于海量数据的聚类分析。

In order to solve the problem of the clustering on massive data under the framework of a centralized system, an optimized algorithm to K-means clustering based on MapReduce is proposed. By using MapReduce parallel programming framework and importing Canopy clustering, this algorithm optimizes initial clustering center, improves communication mode and calculation mode in iteration. The experimental results show that this algorithm can effectively improve the quality of clustering, and can have higher implementation efficiency, its good scalability, thus it fits to clustering analysis on massive data.