基于hadoop的时隙优化任务调度策略研究

A Research of Time Slot Optimization Task Scheduling Strategy Based on the Hadoop

针对分布式云计算平台hadoop任务调度中由于网络时延而导致的调度不平衡问题,提出一种基于时隙优化的任务调度算法。充分参考当前网络条件,将其作为任务调度的重要参考依据,利用SDN对网络带宽的管理能力,根据时隙策略分配带宽,结合局部性原理决定将任务分派到本地还是低负载的其他节点,从全局角度保证任务本地化比例较高的前提下以最优的方式高效分派任务,以提高系统整体的任务处理能力。依据算法设计调度器设计并进行实验,实验结果验证了算法的调度质量。

In this paper, a task scheduling algorithm based on time slot optimization is putted forward to solve the problems of imbalance about the network time delay in scheduling in distributed task scheduling in the cloud com- puting platform hadoop. Referring to the current network conditions, this algorithm as an important reference in task scheduling. According to the strategy of the time slot to allocate bandwidth and using of SDN management ability of network bandwidth, combined with the principle of locality to task assignment whether in local or other nodes of low load, from the global perspective, ensure higher percentage localization task to task in the high efficiency, in the last, the ability to handle task of system as a whole is improved a lot. According to the algorithm to design the scheduler and experimental testing, the results verify the quality of scheduling algorithm.