基于集群规模调整的节能存储策略

Energy-conserving strategies of file storage based on cluster scale adjustment

根据谷歌数据中心研究报告,传统数据中心存在高能耗、低利用率的问题。通过研究集群数据块访问规律,提出一种基于集群规模调整的Hadoop分布式文件系统(HDFS)节能存储策略,实现HDFS高效节能存储。策略主要在集群区域划分、数据块迁移策略优化、缓存机制等方面作出了改进。实验结果表明:使用该节能策略的HDFS比传统HDFS节能35%～40%,其中0.3%的访问需要唤醒服务器,同时引入缓存策略对集群的性能提高了5.1%。

According to the research report of the google data-center,traditional data-center has the problem of the high energy consumption and low utilization ratio.A Hadoop Distributed File System(HDFS)energy-saving storage scheme based on cluster scale adjustment is proposed to realize HDFS efficient energy-saving storage by the access rules of cluster data block in this paper.The strategy is improved in the cluster in the regional division,data block migration strategy optimization and caching mechanisms.The simulation experiment results show that using the energy-saving strategies of HDFS energy-saving varies between35%~40%than traditional HDFS,0.3%access need to wake up the servers.At the same time,while the caching storage is introducted,the performance of the cluster is improved by5.1%compared with traditional HDFS.