一种基于Hadoop的作业转移调度算法

A job transfer scheduling algorithm based on hadoop

云环境服务集群中作业提交存在着非均匀分布的问题,此问题会导致某一时刻作业聚集,从而引起作业的响应时间超出用户容忍范围。针对该问题,笔者在Hadoop平台下利用二级队列技术,提出一种基于队列的作业转移调度策略(JTSA)。实验结果表明,在作业数量某一时刻剧增的情况下,对总完成时间影响不大而且能够较大幅度提高作业的响应速度,随着作业数的增加,这种效果越明显。

The job submission in the cloud cluster exists the non-uniform distribution of aggregate,which could cause the response time over the tolerance range.This paper proposed a schedule algorithm for job transfer,which based on queue strategy by using the two levels queues technology on the Hadoop platform.The test results showed that,the proposed algorithm could reduce the response time without increasing the total elapsed time when the job amount increased sharply.This result could be more obviously with more job amount.