一种基于先验知识动态分配资源的作业调度策略

A Dynamical Scheduling Strategy of Resources Allocation Based on Priory Knowledge

针对目前遥感图像并行处理系统的调度策略中资源分配盲目性,导致作业平均带权周转时间过长和系统利用率低的问题,提出了一种新的基于先验知识的动态分配资源调度策略。这种策略一方面利用先验数据库来存储各种算法单位资源单位数据量的平均运行时间,结合待处理图像的数据量来估算作业单位资源的运行时间,从而为系统作业调度提供准确的并行作业运行时间估计;另一方面利用模糊数学知识动态地计算出集群的整体负载值。最终,本文设计了一种自适应的资源的动态分配方法,可根据作业单位资源的运行时间和当前集群整体系统负载值动态决定作业所需分配的资源数。这种资源分配方式可解决传统资源分配策略在资源分配上的盲目性,缩短作业平均带权周转时间,从而使系统达到负载均衡。通过对比分析和实验结果,本调度策略能较好地解决了目前并行遥感图像处理系统中传统作业调度存在的问题,缩短了系统的作业平均带权周转时间,提高了系统资源利用率,使得整体系统的处理性能得到大幅优化。

The effective job scheduling schemes play a critically important role in the cluster-based parallel processing of remote sensing image.However,the most job scheduling strategies commonly treat various parallel algorithms as undifferentiated jobs without estimating accurate run time of algorithms.Such methods could cause blindness in resources allocation,and always lead to low system utilization and long average weighted turnaround time.To overcome the problems above,a dynamic allocation of resources strategy is proposed in this paper.The unit time（average run time of processing per unit data） of various parallel algorithms that recorded in the knowledge database in advance is used for estimating the accurate run time of real parallel jobs with self-learning ability.Through the accurate estimation of job run time and system load that dynamically computed by fuzzy model,the resources allocation is dynamically decided in order to shorten the average weighted turnaround time,and finally to achieve the load balance of the entire system.Through experimental and comparative analysis,the outstanding scheduling efficiency of the strategy is showed in this paper.