风电场数据中心Hadoop云平台作业调度算法研究

Research on job scheduling algorithm on wind farms data center cloud platform based on Hadoop

风电场数据中心包含状态监测、数据采集等实时类作业和非实时类作业,采用C/S结构存在资源利用率不平衡、管理与维护成本高等缺点。设计了一种基于Hadoop云平台的数据中心架构;针对开源Hadoop平台现有FIFO调度器不能满足实时监测系统要求,在原有FIFO调度器的基础上,设计了一种双队列的作业调度器,综合考虑作业的截止时间和优先级来进行作业调度决策,实验结果表明,与FIFO调度器相比,双队列的作业调度器在集群负载较大时能够表现出较好的性能,保证实时类作业能够优先执行,为风电机组的安全运行提供保障。

Wind data center contains status monitoring, data acquisition real-time jobs and other non-real-time jobs. The structure of C/S has been widely used, but it has many disadvantages, such as the imbalance of resource utilization, the high cost of management and maintenance, etc. This paper presents a data center architecture based on cloud computing platform; in order to solve the FIFO scheduler in the open source Hadoop platform can not satisfy the real-time monitoring jobs＇ requirements, it designs a dual queue job scheduler based on the existing FIFO scheduler, which considers the deadline time and priority of the job to make schedule job decisions. Compared with the FIFO scheduler, the experimental results show that the dual queue job scheduler can make a good performance when the cluster load is large to make the real-time jobs can be executed first, and provides a guarantee for the safe operation of the wind turbines.