基于数据关联性聚类的数据布局算法

Data placement algorithm based on data dependence

现代信息系统的突出特征是基于海量数据的分布式应用集群。优化海量数据的存储布局,以提升存储资源的利用率和应用执行的速度,是一个重要研究课题。由于数据与数据之间存在关联性,只考虑负载均衡的布局算法缺乏实用性,需要进一步考虑数据与数据的关联性以提高应用执行速度。建立了数据和数据的关联矩阵,基于关联矩阵进行聚类,再将数据分配到各个数据中心中,计算执行应用时的数据迁移量,并与一致hash算法进行了比较,结果表明数据迁移量大大低于一致hash算法。

The prominent feature of the modern information systems is the distributed applications clustering based on massive data, so optimizing the storage of mass data to improve the response time of the application service while making full use of the storage resources is an important task. Because of the dependency of the data, the data placement algorithm which simply considers the load balancing is lack of practicability, so it needs to further consider the denpendency of the data to improve the response time of the application service. So it establishes a denpendency matrix, clusters the data based on the dependency matrix, and then distributes the data to each data center. It analyses data movements of the appli-cation and compared with the consistent hashing, the results show that the data movements is greatly decreased.