基于Storm平台的数据迁移合并节能策略

Energy-efficient strategy for data migration and merging in Storm

针对Storm存在低效率、高能耗的问题,通过分析Storm平台的基本框架与拓扑结构,设计了资源约束模型、最优线程数据重组原则和节点降压原则,并在此基础上提出了基于Storm平台的数据迁移合并节能策略(DMM-Storm),包括资源约束算法、数据迁移合并算法和节点降压算法。其中资源约束算法根据资源约束模型,判断工作节点是否允许数据的迁移;数据迁移合并算法根据最优线程数据重组原则,设计了最优的线程数据迁移方法;节点降压算法根据节点降压限制条件,降低了工作节点的电压。实验结果表明,与现有的节能策略相比,执行DMM-Storm在不影响集群性能的前提下,有效降低了能耗。

Storm is suffering the problems of high energy consumption but low efficiency.Aiming at this problem,the resource constraint model,the optimal principle of data reorganization in executors and node voltage reduction principle were proposed based on the analysis of the architecture and topology of Storm,and further the energy-efficient strategy for data migration and merging was put forward in Storm(DMM-Storm),which was composed of resource constraint algorithm,data migration and merging algorithm as well as node voltage reduction algorithm.The resource constraint algorithm estimates whether work nodes are appropriate for data migration according to the resource constraint model.The data migration and merging algorithm designs an optimal method to migrate data according to the the optimal principle of data reorganization in executors.The node voltage reduction algorithm reduces voltage of work nodes according to node voltage reduction principle.The experimental results show that the DMM-Storm can reduce energy consumption efficiently without affecting the performance of cluster compared with the existing researches.