期刊文献+

基于SDN的MapReduce调度优化方法 被引量:1

MapReduce Scheduling Optimization Method Based on SDN
下载PDF
导出
摘要 针对传统的面向终端的粗粒度流量控制方法易引起数据中心网络拥塞问题,将软件定义网络(SDN)引入大数据应用体系架构,提出了基于SDN的MapReduce调度优化方案。该方案中,SDN控制器可由多个MapReduce应用程序共享,并根据控制(master)节点的需求对网络进行拓扑管理和资源调度,降低了网络拥塞发生的概率,同时提升了MapReduce计算速度。 The traditional data transmission methods, which adjust the protocol parameters and transmission paths based on network state estimation, can result in the data center network con- gestion. The softwawre-defined network (SDN) is introduced to the big data application frame- work. A MapReduce scheduling optimization method based on SDN is proposed. Several MapRe- duce applications can share a SDN controller in the method. The SDN controller can allocate the resource and the manage physical connectivity based on requirements of master nodes. There- fore, the network congestion can be alleviated and the computation speed of MapReduce can be improved.
出处 《指挥信息系统与技术》 2015年第2期49-52,共4页 Command Information System and Technology
关键词 大数据 软件定义网络 MAPREDUCE框架 Hadoop框架 big data softwawre-defined network MapReduce framework Hadoop framework
  • 相关文献

参考文献7

  • 1White T. Hadoop: the definitive guide[M]. [S. 1. ]: O'Reily Media Incroporated, 2009.
  • 2Isard M, Budiu M, Yu Y, et al. Distributed data-par- allel programs from sequential building blocks [C] // Proceedings of ACM EurySys' 07. Lisboa: ACM, 2007.
  • 3Zaharia M, Chowdhury M, Franklin M J, et al. Spark: cluster computing with working sets[C]//Pro- ceedings of USENIX Hot Cloud' 10. Boston: USE- NIX, 2010.
  • 4Apache HBase [EB/OL]. [2015-01-30]. http:// hbase, apache, org.
  • 5Weigle E, Feng W. A comparison of TCP automatic tuning techniques for distributed computing[C] // Pro- ceedings of llth IEEE International Symposium on High Performance Distributed Computing. [S. l.]: IEEE, 2002.
  • 6Seedorf J, Burger E. Application-layer traffic optimi- zation (alto) problem statement, RFC-5693[R/OL]. 1-2015-01-30]. http: //tools, ietf. org/htm/rfc5693.
  • 7Chandra P, Fisher A, Kosak C, et al. Darwin: Re- source management for value-added customizable net- work service [J]. IEEE Network: the Magazine of Global Internetworking, 2001,15( 1 ) : 22-35.

引证文献1

二级引证文献3

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部