基于云计算的并行k-means算法研究

Research on parallel k-means algorithm based on cloud computing

针对传统k-means聚类算法面对海量数据存在时间复杂度急剧增加的问题,结合云计算的优势,提出基于MapReduce编程框架来实现k-means聚类算法的并行化处理。Map函数完成每个样本记录到聚类中心的距离计算并标记其所属聚类类别,Reduce函数汇总中间结果并计算出新的聚类中心,供下一轮迭代使用。通过实验表明:基于MapReduce的并行化k-means聚类算法具有较好的加速比和良好的扩展性。

For the problem of high time complexity in dealing with huge data of k-means algorithm,propose a parallel method using MapReduce programming model and cloud computing to reduce the time complexity of k-means. The distance between each record and each cluster was calculated and new category was marked to each record in the Map function.All the records of the same key value were sent to a single reducer and get the new cluster centroids for the next MapReduce Job.Experimental results show that the parallel k-means algorithm based on MapReduce has basically linear speedup with an increasing number of node computers and good scalability.