一种采用消息模型的多集群作业管理方案设计

Design of a Multi-cluster Job Management Scheme Using Message Model

文中针对多集群环境资源异构且地域分散、网络环境不可靠以及面向用户需求的特点,提出了一种采用消息模型的多集群作业管理方案。该方案采用全局-局部的层次调度方法,基于发布-订阅的消息模型,根据当前网络环境、用户作业的资源需求、各集群自身负载情况进行综合统一调度管理。实践证明,采用该方案设计实现的多集群作业管理系统实现了多集群环境下的资源监控、资源管理、作业调度、作业控制、数据管理等功能,有效解决了在资源异构及网络环境不可靠条件下的系统稳定性问题,显著提高了多集群系统作业吞吐能力。

Aiming at the features of resources heterogeneity, geographical dispersion and unreliable network environment in multi-cluster environment,propose a method to manage job among multi-clusters using message module, which uses global-local two levels scheduling. It adopts publish-subscribe message model and can unify management and schedule job based on the present network environment, job＇ s resource requirements and the various cluster loads. Practice has proved that the multi-cluster job management system which is de- signed and implemented by this method has achieved multi-cluster environment resource monitoring,resource management,job scheduling ,job control, data management. It solves the stability problem effectively under the heterogeneous resources and unreliable network environment conditions. Also it significantly improves the throughput capacity of multi-cluster system.