期刊文献+

一种Hadoop Yarn的资源调度方法研究 被引量:18

A Study on Scheduling Method of Hadoop Yarn
下载PDF
导出
摘要 针对Hadoop Yarn资源调度问题,为提高集群作业执行效率,提出一种基于蚁群算法与粒子群算法的自适应Hadoop资源调度算法SRSAPH.SRSAPH中,通过Hadoop Yarn跳通信机制获取负载、内存、CPU速度等属性信息初始化信息素矩阵;同时,将粒子群算法的自我认知能力与社会认知能力引入到蚁群算法,提高算法的收敛速度;此外,根据蚁群算法全局最优解的波动趋势动态调整信息素挥发系数,提高解的精度.实验表明,采用SRSAPH进行资源调度,集群的作业执行时间缩短至少10%. In view of the resource scheduling problem of Hadoop Yam, to improve the execution efficiency of the cluster job, we propose a Self-adapt Resource Scheduling algorithm based on Ant Colony Algorithm and Particle Swarm Algorithm in Hadoop (SRSAPH). In SRSAPH, we initialize the pheromone matrix of SRSAPH by using the attribute information of load, memory, and CPU speed obtained through the heartbeat message transfer mechanism. Meanwhile, we introduce the self-cognitive ability and social cognition ability of particle swarm algorithm into the ant colony algorithm to speed up the rate of convergence of the algorithm. Moreover, we dynamically adjust the pheromone evaporation rate based on the fluctuation trends of global optimal solution to enhance the accuracy of the solutions. Experimental result shows that by using SR- SAPH in resource scheduling,the execution time of cluster job is shorten by 10%.
出处 《电子学报》 EI CAS CSCD 北大核心 2016年第5期1017-1024,共8页 Acta Electronica Sinica
基金 国家自然科学基金(No.41301407) 江苏省自然科学基金(No.BK20130819)
关键词 资源调度 蚁群算法 粒子群算法 HADOOP YARN resource scheduling ant colony algorithm particle swarm algorithm Hadoop Yam
  • 引文网络
  • 相关文献

参考文献21

  • 1Elghoneimy E, Bouhali O, Alnuweiri H. Resource alloca- tion and scheduling in cloud computing[ A ]. 2012 Interna- tional Conference on Computing, Networking and Commu- nications [ C ]. Hawaii: IEEE,2012. 309 - 314.
  • 2Gokilavani M, SelviS, Udhayakumar C. A survey on re- source allocation and task scheduling algorithms in cloud environment[ J]. International Journal of Engineering and Innovative Technology ,2013,3 (4) : 173 - 179.
  • 3Chauhan J. Simulation and performance evaluation of ha- doop capacity scheduler [ D ]. Saskatoon: University of Saskatchewan,2013:93 - 96.
  • 4Fair Scheduler Guide [R/OL]. http://hadoop, apache. org/commort/docs/r0. 20. 2/fair _ shcehduler, html, 2013 -08 -04.
  • 5Rasooli A, Down D G. A hybrid scheduling approach for scalable heterogeneous hadoop systems [ A ]. 2012 High Performance Computing, Networking, Storage and Analysis [C]. Salt Lake City:IEEE,2012. 1284 -1291.
  • 6Rasooli A, Down D G. COSHH:A classification and opti- mization based scheduler for heterogeneous Hadoop sys- tems[ J ]. Future Generation Computer Systems, 2014,36 (7) :1 -15.
  • 7Yazir Y O, Matthews C, Farahbod R, et al. Dynamic re- source allocation in computing clouds using distributed multiple criteria decision analysis[ A]. 3th IEEE 2010 Inter- national Conference on Cloud Computing [ C ]. Florida: IEEE,2010.91 -98.
  • 8Yue Z, Xu Q. Resource allocation and scheduling theory based on distributed environment [ A ]. 16^th International Conference on Advanced Communication Technology [ C ]. PyeongChang Korea: IEEE, 2014. 1124 - 1128.
  • 9Ergu D, Kou G, Peng Y, et al. The analytic hierarchy process:task scheduling and resource allocation in cloud computing environment [ J ]. The Journal of Supercomput- ing,2013,64(3) :835 - 848.
  • 10Senouci A B, Eldin N N. Use of genetic algorithms in re- source scheduling of construction projects [ J ]. Journal of Construction Engineering and Management, 2004, 130 (6) : 869 - 877.

同被引文献138

引证文献18

二级引证文献69

;
使用帮助 返回顶部