基于Hadoop改进的云任务调度算法研究

Research on an Improved Cloud Task Scheduling Algorithms Based on Hadoop

在云环境下,通常需要处理大量的计算任务,云平台中的任务调度策略直接关系到云计算系统的运算性能,而Hadoop计算框架以可靠、容错的方式可以在大型集群上并行处理大量数据,YARN是Hadoop集群的资源管理系统,在分析Hadoop YARN的资源调度机制的基础上,提出了一种面向任务完成时间为需求的调度算法,对任务进行动态调度,以满足所有任务的完成时间需求.

In a cloud environment,a large number of computing tasks are usually processed.The task scheduling strategy in the cloud platform is directly related to the computing performance of the cloud computing system.The Hadoop computing framework can process large amounts of data in parallel on a large cluster in a reliable,fault-tolerant manner.YARN is the resource management system for Hadoop clusters.Based on the analysis of the resource scheduling mechanism of Hadoop YARN,this paper proposes a scheduling algorithm that is oriented to the task completion time and dynamically schedules tasks to meet the completion time requirements of all tasks.