大数据处理集群面向效用最大化的动态资源分配技术研究

Study on the dynamical resource scheduling approach for utility maximization in big data cluster

文章引入时间—效用函数表述任务完成时间与收益的关系,从而准确刻画不同类型大数据处理任务的需求。为最大化系统效用,设计了优先关系调度机制,确定资源分配。模拟实验表明,提出的优先关系调度机制比公平调度机制的效用提升超过50%。

In this paper,we introduce time-utility function to describe the relationship between task completion time and utility,thus accurately describe the requirements of different types of large data processing tasks.In order to maximize the utility of the system,a priority relation scheduling mechanism is designed to determine the resource allocation.The simulation results show that the proposed priority relation scheduling mechanism achieves more than 50%utility than the fair scheduling mechanism.