基于多代表点的大规模数据模糊聚类算法

Fuzzy clustering algorithm based on multiple medoids for large-scale data

针对传统模糊聚类在大规模数据场景下,由于内存的限制不能一次装载所有数据,以及在通过聚类捕捉数据的潜在结构和描述各个类时仅使用单个代表点存在信息量不足的问题,提出一种基于多代表点的大规模数据模糊聚类算法.该算法通过对大规模数据进行分块,在对每个数据块进行聚类时使用多个代表点描述捕捉到的数据的潜在结构和各个类信息,并通过考虑代表点与代表点之间在聚类过程中的约束关系,提高最后聚类结果的精度.在模拟数据集和真实数据集上的3组实验验证了所提出算法的有效性.

For the problem that the traditional fuzzy clustering is not able to load all the data at a time because of the limited memory in the application scenario for large-scale data, and using a single medoid is insufficient to capture the underlying structure of data and describe each cluster, a fuzzy clustering algorithm based on multiple medoids for large-scale data is presented. The algorithm handles data chunk by chunk, and uses multiple medoids to represent the underlying data structure and each cluster information in one chunk, and the pairwise constraints from the relationship between two identified medoids are taken into account. These mechanisms improve the accuracy of the final clustering results. The effectiveness of the proposed algorithm is verified by three sets of experiments on a simulated dataset and two real datasets.