双MapReduce改进的Canopy-Kmeans算法

Improved Canopy-Kmeans Algorithm based on Double-MapReduce

由于传统的Canopy-Kmeans算法在中心点的选取存在随机性,其迭代过程的冗余计算降低了算法的运行效率.文中基于＂最小最大原则＂和三角不等式原理,在Hadoop平台上提出了一种基于双MapReduce改进的Canopy-Kmeans算法.实验结果表明：设计的并行算法精确率在不同大小的数据集上平均提高了15.3%,加速比和扩展性随着数据规模和节点的不断增加也相应的提高了1.5~3倍,解决了Canopy中心点选中存在的问题和迭代过程中冗余的距离计算.

The Canopy-Kmeans algorithm has the disadvantage of great randomness in the selection of center points,and the redundant computation in the iterative process significantly reduces the operation efficiency of the algorithm.So the paper proposes an improved Canopy-Kmeans algorithm based on the Double-MapReduce on the Hadoop platform,which is based on the ＂ minimum maximum principle＂ and the principle of triangle inequality.The experimental results show that the precision of the designed parallel algorithm is raised by 15.3% on average,and the speedup and scalability are increased by 1.5to3 times with the increase of the data size and the number of node.The problem existing in the selection of Canopy center point is successfully solved and the redundant distance calculation in iterative is avoided.