一种改进的Hadoop多用户作业调度方法

Improved method in solving Hadoop multi-user scheduling

针对目前Hadoop作业调度方法服务水平不高、资源利用率低的问题,提出了一种改进的Hadoop多用户作业调度算法。分析了Hadoop现行调度算法存在的不足,提出了基于服务质量(Qo S)的作业选择量化和基于遗传算法的任务选择均衡化的方法,最后采用Hadoop平台对算法进行了仿真。仿真结果表明,该资源调度方法提高了作业的服务质量,实现了资源的合理调度。

Aiming at the job scheduling method of Hadoop operation service level is not high,and the low utilization rate of resources problem,this paper proposed an improved Hadoop multiuser scheduling algorithm. Firstly,it analyzed the shortcomings existed in Hadoop scheduling algorithm. Then put forward a job selection method based on the quality of service,and a task selection equalization method based on genetic algorithm. Finally,it simulated the algorithm by using the Hadoop platform. The simulation results show that,the resource scheduling method improves the operation quality of service,to achieve a reasonable scheduling of resources.