MapReduce下融合PAM算法与仔细播种的多样本归并聚类

Multi-samples Merging Clustering Algorithms Combining PAM Algorithm and Careful Seeding Based on MapReduce

传统PAM(Partitioning Around Medoids)算法时间复杂度较高,处理大数据集时效率低下.近年来,越来越多研究者使用MapReduce模型来使聚类算法获得更高的性能,然而MapReduce模型在算法迭代过程中需要多次重启任务、从文件系统读取数据和数据洗牌,影响数据处理效率.本文提出两种基于MapReduce的融合PAM算法与仔细播种的聚类处理模型,在保持PAM算法聚类有效性的同时,在算法性能上获得显著提高.性能试验和聚类有效性实验的结果表明本文提出的方法达到了预期的效果且具有很好的可扩展性.

Common PAM （Partitioning Around Modoids ） algorithm works inefficiently for large-scale data set due to its time complexity. Recently,more and more researchers apply MapReduce model to obtain high performance for clustering algorithms. However,MapReduce model needs repeated times of restarting jobs ,reading data from file system and data shuffling which will have impacts on data processing efficiency. In this paper, we propose two clustering processing models based on MapReduce model, PAM algorithm and careful seeding to obtain high performance and maintain cluster validity of PAM in the same time. The performance evaluation and clustering validation experiments demonstrate that the methods we have proposed are efficient, robust and scalable.