期刊文献+
任意字段
题名或关键词
题名
关键词
文摘
作者
第一作者
机构
刊名
分类号
参考文献
作者简介
基金资助
栏目信息

基于LATE的Hadoop数据局部性改进调度算法 被引量：17

New Improvement of the Hadoop Relevant Data Locality Scheduling Algorithm Based on LATE
下载PDF
导出
摘要 调度问题是目前云计算研究中的热点问题,其目的是如何协同云计算资源,使其得到充分合理的利用。数据局部性是特定云平台Hadoop的主要特性之一。针对该特性,在Hadoop原有调度算法LATE的基础上提出了一种基于数据局部性的改进算法,以解决数据局部性带来的慢任务备份执行时读取数据要占用大部分时间而影响其处理速率的问题。最后,对该算法进行了实验及性能分析,并验证了算法在提高任务的响应时间和整个系统吞吐率方面有很大改进。 In the present,scheduling problem is a hot cloud computation research issues,and the purpose is to coordinate the cloud computation resources to be fully rational use.Data locality is one of the main properties in the particular cloud platform for Hadoop.The paper discussed the property,proposed a new improvement of the Hadoop relevant data locality scheduling algorithm based on LATE.The algorithm mainly soves the bakeup of slow task performance pro-blem which arises during the implementation of data read,taking most of the time and envently influencing its processing speed.Finally,carried on experiment to the algorithm and analyzed the funcation,verified the algorithm to improve the response time and the whole system throughput.
作者 李丽英 唐卓 李仁发
机构地区 湖南大学计算与通信学院
出处 《计算机科学》 CSCD 北大核心 2011年第11期67-70,共4页 Computer Science
基金 国家工信部核高基项目(2009ZX01038-001)资助
关键词 HADOOP MAPREDUCE LATE 数据局部性 Hadoop MapReduce LATE Date locality
分类号 TP393.03 [自动化与计算机技术—计算机应用技术]
  • 相关文献

参考文献15

  • 1Vaquero L M,Rodero-Merino I.,Caceres J, el al. A break in the cloud: Towards a Cloud Definition [J].ACM SIGCOMM Computer Communication Review, 2009,39 ( 1 ) : 50-55.
  • 2Vaquero L M,Rodero-Merino L,Caceres J, et al. A break in the cloud: Towards a Ckoud Definition[J].ACM SIGCOMM Computer Communication Review, 2009,39 ( 1 ) : 50-45.
  • 3Crovella M, HarchobBalter M, Murta C D. Tasassignment in a distributed system:Improving performance by unbalancing load [M]. Measurement and Modeling of Computer Systems, 1998: 268-269.
  • 4Dean J,Ghemawat S. MapReduce: Simplified Data Processing on Large Clusters[J].Commun. ACM, 2008,51 ( 1 ) : 107-113.
  • 5Huston L, Sukthankar R, Wiekremesinghe R, et al. Diamond: A storage architecture for early discard in interactive search[C]// Proceedings of the 2004 USENIX File and Storage Technologies FAST Conference. April 2004.
  • 6Riedel E,Faloutsos C,Gibson G A, et al. Active disks for largescale data processing[C]//IEEE Computer. June 2001:68-74.
  • 7Thain D, Tannenbaum T, Livny M. Distributed computing in practice:the Condor experience[J].Concurrency and Computa tion-Praetiee and Experience, 2005,17 (2-4) : 323-356.
  • 8Zahafia M,Konwinski A,Joseph A. Improving MapReduce Per formance in Heterogeneous Environments [C]//Proc of the 8th Usenix Syrup on Operating Systems Design and Implementa tion. 2008 : 29-42.
  • 9HADOOP-3759: Provide ability to run memory intensive jobs without affecting other running tasks on the nodes[EB/OL]. https://issues, apache, org/jira/browse/HADOOP 3759.
  • 10Zaharia M, Borthakur D, Sarma J S, et al. Job Scheduling for Multi-user MapReduce Clusters[R]. EECS-2009 55. April 2009.

二级参考文献10

  • 1Vaquero L M, Rodero-Merino L, Caceres J, et al. A Break in the Clouds: Towards a Cloud DefinitionD]. ACM SIGCOMM Computer Communication Review, 2009, 39 ( 1 ) : 50- 55.
  • 2Bryant R E. Data-Intensive Supercomputing: the Case for DISC[R]. CMU Technical Report CMU-CS-07-128, Department of Computer Science, Carnegie Mellon University, 2007.
  • 3Dean J, Ghemawat S. MapReduce: Simplied Data Processing on Large Clusters[C]//Proc of OSDI '04,2004 : 137-150.
  • 4Colbyranger, Raghuraman R, Penmetsa A. Evaluating MapReduce for Multi-Core and Multiprocessor Systems[C]//Proc of the IEEE 13th Int'l Syrup on High Performance Computer Architecture, 2007 : 13-24.
  • 5Kruijf M D, Sankaralingam K. MapReduce for the Cell B. E. Architecture[-R]. Technical Report CS-TR-2007-1625, University of Wisconsin Computer Sciences University of Wisconsin, 2007.
  • 6He B S, Fang W B, Luo Q, et al. Mars: A MapReduce Framework on Graphics Processors[C]//Proc of the 17th Int'l Conf on Parallel Architectures and Compilation Techniques, 2008 : 260-269.
  • 7Apache Hadoop. Hadoop [EB/OL]. [2009-03-06]. http://hadoop, apache, org/.
  • 8Yahoo. Yahoo! Hadoop Tutorial [EB/OL]. [2009-02-27]. http:// public, yahoo, com/gogate/hadoop-tutorial/start-tutorial, html.
  • 9Ghemawat S, Gogioff H, Leung P T. The Google File System[C]//Proc of the 19th ACM Syrnp on Operating Systems Principles, 2003 : 29-43.
  • 10Zaharia M, Konwinski A, Joseph A D. Improving MapReduce Performance in Heterogeneous Environments [C]//Proc of the 8th Usenix Syrup on Operating Systems Design and Implementation, 2008 : 29-42.

共引文献20

同被引文献163

引证文献17

二级引证文献111

计算机科学
2011年 第11期

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部