基于Hadoop平台上面向电影数据集Kmeans算法的改进

The Improvement of Kmeans Algorithm Facing the Movie Dataset Based on Hadoop Platform

针对聚类算法并行化的需求,该文对基于Hadoop平台Kmeans算法进行了改进,选用Canopy算法对数据进行预处理,并在具有一定数据结构的电影数据集上进行了单机对比实验,集群加速比实验和集群扩展率实验,分别体现改进后算法实现的高效性、良好的加速比和可扩展性,从而可以有效地运用在实际海量数据挖掘中.

According to parallelism demand of the clustering algorithm, This paper improved the implemention of the kmeans algorithm based on the Hadoop platform. We do the preprocess on the dataset using the canopy algorithm, and conduct the single contrast experiment, cluster speed up experiment and cluster expansion rate experiment, showing the high effiency, better speed up and scalability, thus the implemention can be used in the pratical mass data mining effectively.