数据密集作业在GPU集群上的调度算法研究

Algorithm of Scheduling for Data-intensive Computing Operations onto GPU Cluster

数据密集型作业包含大量的任务,使用GPU设备来提高任务的性能是目前的主要手段.但是,在解决数据密集型作业之间的GPU资源公平共享以及降低任务所需数据在网络间的传输代价方面,现有的研究方法没有综合考虑资源公平与数据传输代价的矛盾.分析了GPU集群资源调度的特点,提出了一种基于最小代价最大任务数的GPU集群资源调度算法,解决了GPU资源的公平分配与数据传输代价较高的矛盾.将调度过程分为两个阶段:第1阶段为各个作业按照数据传输代价给出自己的最优方案;第2阶段为资源分配器合并各个作业的方案,按照公平性给出全局的最优方案.首先,给出了GPU集群资源调度框架的总体结构,各个作业给出自己的最优方案,资源分配进行全局优化;第二,给出了网络带宽估计策略以及计算任务的数据传输代价的方法;第三,给出了基于GPU数量的资源公平分配的基本算法;第四,提出了最小代价最大任务数的资源调度算法,描述了资源非抢夺、抢夺以及不考虑资源公平策略的实现策略;最后,设计了6种数据密集型计算作业,对所提出的算法进行了实验.通过实验验证,最小代价最大任务数的资源调度算法对于资源公平性能够达到90%左右,同时亦能保证作业并行运行时间最小.

Data-intensive tasks include a large number of tasks.Using GPU devices to improve the performance of tasks is the main method currently.However,in the case of solving the fair sharing of GPU resources between data-intensive tasks and reducing the cost of data network transmission,the existing research methods do not comprehensively consider the contradiction between resource fairness and data transmission costs.The study analyzes the characteristics of GPU cluster resource scheduling,and proposes an algorithm based on the minimum cost and the maximum number of tasks in GPU cluster resource scheduling.The method can solve the contradiction between the fair allocation of GPU resources and the high cost of data transmission.The scheduling process is divided into two stages.In the first stage,each job gives its own optimal plan according to the data transmission costs,and in the second stage,the resource allocator merges the plan of each job.Firstly,the study gives the overall structure of the framework,and the source allocator works globally after each job giving its own optimal plan.Secondly,the network bandwidth estimation strategy and the method of computing the data transmission cost of the task are given.Thirdly,the basic algorithm for the fair allocation of resources based on the number of GPUs is given.Fourthly,the scheduling algorithm with the smallest cost and the largest number of tasks is proposed,which describing the implementation strategies of resource non-grabbing,robbing and resource fairness strategies.Finally,six data-intensive computing tasks are designed,and the algorithm proposed in the study is tested,and the experiments verifies that the scheduling algorithm can achieve about 90%of resource fairness,while also ensuring that the parallel operation time of jobs is minimized.