局部数据位置感知的云计算工作流调度优化

Workflow Scheduling Optimization Based on Local Data Location Aware in Cloud Computing

针对高性能计算(High Performance Computing, HPC)云环境中数据密集型工作流的调度效率较低问题,提出一种基于局部数据位置感知资源管理的调度方法,以网络宽带为载体将数据位置和数据传输时间应用于工作流任务调度中,同时平衡节点级任务的资源使用性和并行性,设计与局部性数据布局和传输相关的工作流模型,采用虚拟机(VirtualMachine,VM)并行处理数据流任务。利用云环境特有的公平性准则,通过实验验证了所提方法可以在提高资源利用率的同时,大幅度改善工作流的调度效率。

In this paper, a novel workflow scheduling method based on local data location aware and resource management is proposed for low scheduling efficiency of the data-intensive workflows in high performance computing(HPC) cloud environments. The data-position and data transfer time are applied to workflow scheduling based on network bandwidth and the resource utilization and parallelism of tasks are also balanced at the node-level. In addition, a workflow model related to local data layout and transmission is designed, and the virtual machine(VM) is used to process the data stream task in parallel. The proposed method is validated based on fairness criterion of cloud environments, and experimental results indicate that the proposed method can not only enhance resource utilization ratio, but also improve scheduling efficiency of data-intensive workflows substantially.