摘要
MapReduce Job的调度机制一直是学术研究的热点。在分析MapReduce数据流调度模型的基础上,提出一种面向MapReduce数据流的公平调度方法FlowS。该方法采用数据流池来分配资源以保证MapReduce数据流的隔离性,并且采用数据流池动态构建算法来确保资源的公平分配。实验表明,该调度方法可以有效提高Hadoop集群对MapReduce数据流的处理效率。
MapReduce Job scheduling has been paid great attention in academic research.Based on the analysis of MapReduce dataflow scheduling model,this paper presented a fair scheduling method for MapReduce dataflow-FlowS.This method can not only provide the isolation of MapReduce dataflow through dataflow pools,but also assure the fairness of resource allocation through dynamic construction algorithm.The results of experiences show that the proposed method can improve the processing efficiency of Hadoop Clusters.
出处
《计算机科学》
CSCD
北大核心
2012年第9期157-161,174,共6页
Computer Science
基金
国家自然科学基金项目(61170074)
国家科技重大核高基项目(2010ZX01042-001-001-05)
国家科技支撑计划(2011BAH15B05
2012BAH05F02)资助
