期刊文献+

Hadoop远程过程调用机制的分析和应用 被引量:10

Analysis and application of mechanism of Hadoop RPC communiucation
下载PDF
导出
摘要 为了高效快速的开发和使用Hadoop远程通信功能,对Hadoop远程过程调用(Remote Procedure Call,RPC)机制进行分析,提出独立调用Hadoop RPC实现多客户端并发通信和大数据传输的机制,既继承Hadoop PRC框架,又在通信中增加数据文件传输,并采用线程池模拟多客户线程的测试方案。通过测试发现Hadoop远程过程调用在通信数据负载量和数据传输量的大致范围,测试方案和结论可用于Hadoop RPC应用的配置和开发。 In order to develop and use Hadoop remote communication efficiently and rapidly,the program is designed to implement multi-client and data transmission after analyzing Hadoop RPC(Remote Procedure Call) mechanism.It extends the Hadoop RPC framework,transfers data files and simulates multi-client threads using thread pool.The report is given to reveal the range of the data load and transmission of remote communication according to Hadoop RPC test.Test program and conclusions can be used in distributed applications and configuration by using Hadoop RPC.
作者 王博 陈莉君
出处 《西安邮电学院学报》 2012年第6期74-77,共4页 Journal of Xi'an Institute of Posts and Telecommunications
基金 陕西省教育厅自然科学基金资助项目(11JK1069)
关键词 Hadoop框架 远程过程调用(RPC) 分布式存储 分布式计算 并发 Hadoop framework Remote Procedure Call distributed storage distributed computing concurrency
  • 相关文献

参考文献8

二级参考文献38

  • 1高洁,吉根林.文本分类技术研究[J].计算机应用研究,2004,21(7):28-30. 被引量:36
  • 2WHITE T.Hadoop,the definitive guide[M].O'Reilly Media,Inc,2009.
  • 3DEAN J,GHEMAWAT S.MapReduee:simplified data processing on large clusters.[C]//Proc of the 6th Symposium on Operating Systems Design and Implementation.San Francisco:Google Inc,2004.
  • 4Hadoop官方文档:http://hadoop.apache.org/common/docs/r0.18.2/cn/mapred_tutorial.html,2008.
  • 5Hadoop官方网站.2010.http://hadoop.apache.org.
  • 6HDFS Architecture. 2010. http://hadoop.apache.org/common/ docs/current/hdfs_design.html.
  • 7Borthakur D. The Hadoop Distributed File System: Architecture and Design. Apache Software Foundation. 2007, 3-14.
  • 8Dean J, Ghemawat S. Mapredteuce: Simplified Data Processing on Large Clusters. Communications of ACM,2008, 51(1):107-113.
  • 9Ajkumarbuyya SP, Vecchiola C. Cloudbus Toolkit for Market-Oriented Cloud Computing. Lecture Notes in Computer Science, 2009,5931 (2009):24-44.
  • 10Hadoop Cluster Setup. 2010. http://hadoop.apache.org/ commorddocs/current/cluster_setup.

共引文献143

同被引文献44

  • 1金澈清,钱卫宁,周傲英.流数据分析与管理综述[J].软件学报,2004,15(8):1172-1181. 被引量:161
  • 2王春梅.基于数据仓库的数据挖掘技术[J].西安邮电学院学报,2006,11(5):99-102. 被引量:6
  • 3崔小燕.Linux集群系统分析[J].西安邮电学院学报,2006,11(5):103-106. 被引量:13
  • 4杨家海;吴建平;安常青.互联网络测量理论与应用[M]北京:人民邮电出版社,2009178-196.
  • 5Morariu C,Racz P,Stiller B. SCRIPT:A framework for scalable real-time IP flow record analysis[A].Piscataway,NJ:IEEE,2010.278-285.
  • 6Youngseok L,Wonchul K,Hyeongu S. An Internet traffic analysis method with MapReduce[A].Piscataway,NJ:IEEE,2010.357-361.
  • 7Deri L,Chou E,Cherian Z. Increasing data center network visibility with cisco NetFlow-Lite[A].Piscataway,NJ:IEEE,2011.1-6.
  • 8Borthakur D. HDFS architecture guide[EB/OL].http://hadoop.apache.org/docs/r1.0.4/hdfs_design.html,2013.
  • 9Dean J,Ghemawat S. MapReduce:simplified data processing on large clusters[J].Communications of the ACM,2008,(01):107-113.doi:10.1145/1327452.1327492.
  • 10Apache. MapReduce tutorial[EB/OL].http://hadoop.apache.org/docs/r1.0.4/mapred_tutorial.html,2013.

引证文献10

二级引证文献60

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部