面向MapReduce异构集群的低功耗调度技术研究

RESEARCH ON LOW POWER SCHEDULING TECHONOLOGY FOR MAPREDUCE HETEROGENEOUS CLUSTER

随着网络数据规模的急剧膨胀,集群在数据处理中扮演日益重要的角色。然而集群中任务的不合理分配导致能量利用率很低。面向数据处理领域中最常用的MapReduce异构集群,通过对节点上的任务进行调度,在保证处理时间限制的前提下最小化能耗。将该问题模型化并基于CPLEX优化工具进行求解。使用GridSim模拟器,以TeraSort和K-means作为评测对象对算法效果进行了评测。实验结果显示,与标准FIFO调度策略相比,所提出的方法使得平均功耗下降了29%。

With the sharp expansion of Internet data,cluster plays a more crucial role in data processing. However,unreasonable assignment of tasks results in high energy consumption. Aiming at the MapReduce heterogeneous cluster,we devote to minimize the energy consumption with the constraint of the guaranteed processing time by reasonable task scheduling. We formulated the problem and solved it based on the CPLEX optimization tool. To evaluate its effect,two typical applications,TeraSort and K-means clustering,were tested based on the Grid Sim. Compared with that of the typical FIFO scheduling strategy,the energy consumption of scheduling strategy proposed is decreased by 29%.