期刊文献+

一种大数据放置方法 被引量:3

A kind of Big Data Placement Method
下载PDF
导出
摘要 数据密集型应用越来越多,如何将大数据在数据中心实现有效放置变得日益重要。研究了大数据的放置模型。影响大数据放置的因素主要有:能耗、异构节点的服务能力及具有关联计算的数据集。基于这3个因素设计了一种节能、充分考虑异构节点服务能力及提升MapReduce处理Join连接的效率的大数据放置模型。该模型将有效实现大数据的有效放置管理,同时也为未来软件定制数据中心奠定了基础。 More and more data-intensive applications have come into being.It is becoming more and more important for the big data' efficient placement in the data center.This paper proposed a kind of big data placement model.The major factors that influce the big data placement have the following three points:energy consumption,sevice capability of heterogeneous node and the data sets which have associated computing.Based on these three factors,our big data placement model considers the energy-saving,service capability of heterogeneous node and the complex Join query mapreduce computing so on.This model can implement the big data's efficient placement management efficiently.At the same time,it will estabilish a foundation for software customized data center in the future.
作者 张桂刚
出处 《计算机科学》 CSCD 北大核心 2014年第6期1-4,36,共5页 Computer Science
基金 高等学校博士学科点专项科研基金课题(20100002110082)资助
关键词 大数据 数据放置 节能 异构节点 关联计算 Big data Data placement Energy-saving Heterogeneous nodes Associated computing
  • 相关文献

参考文献10

  • 1覃雄派,王会举,李芙蓉,李翠平,陈红,周烜,杜小勇,王珊.数据管理技术的新格局[J].软件学报,2013,24(2):175-197. 被引量:110
  • 2Kaushik R T,Bhandarkar M.GreenHDFS:towards an energyconserving,storage-efficient,hybrid Hadoop compute duster[C]//HotPower' 10 Proceedings of the 2010 International Conference on Power Aware Computing and Systems.2010:1-5.
  • 3Arasanal R M,Rumani D U.Improving MapReduce Performance through Complexity and Performance Based Data Placement in Heterogeneous Hadoop Clusters[C]//Distributed Computing and Internet Technology,Lecture Notes in Computer Science.2013,7753:115-125.
  • 4Xie J,Yin S,Ruan X,et al.Improving MapReduce performance through data placement in heterogeneous Hadoop clusters[C]// Proceedings of IPDPS Workshops.2010:1-9.
  • 5王俊伟.大规模多媒体存储系统中数据放置与调度策略的研究[D].长沙:国防科学技术大学,2005.
  • 6林伟伟.一种改进的Hadoop数据放置策略[J].华南理工大学学报(自然科学版),2012,40(1):152-158. 被引量:35
  • 7Mohamed Y,Tian Yuan-yuan,(O)zcan F,et al.CoHadoop:flexible data placement and its exploitation in Hadoop[J].Proceedings of the VLDB Endowment VLDB2011,2011,4 (9):575-585.
  • 8Dittrich J,Quian'e-Ruiz J A,Jindal A,et al.Hadoop++:Making a yellow elephant run like a cheetah (without it even noticing)[J].PVLDB2010,2010,3(1/2):518-529.
  • 9Zhao Yan-rong,Wang Wei-ping,Meng Dan,et al.A data locality optimization algorithm for large-scale data processing in Hadoop[C]//ISCC 2012.2010:655-661.
  • 10赵彦荣,王伟平,孟丹,张书彬,李均.基于Hadoop的高效连接查询处理算法CHMJ[J].软件学报,2012,23(8):2032-2041. 被引量:36

二级参考文献144

  • 1Rajkumar Buyya,Chee Shin Yeo,Srikumar Venugopal,et al.Cloud computing and emerging IT platforms:vision,hype,and reality for delivering computing as the 5 th utility[J].Future Generation Computer Systems,2009,25 (6):599-616.
  • 2Dean J,Ghemawat S.MapReduce:simplified data processing on large clusters[J].Communications of the ACM,2008,51 (1):107-113.
  • 3Borthakur D.Hadoop[EB/OL].[2011-06-15].http://lucene.apache.org/hadoop.
  • 4Ghemawat S,Gogioff H,Leung P T.The google file system[C] //Proc of the 19 th ACM Symp on Operating Systems Principles.New York:ACM,2003:29-43.
  • 5John Dorion.Applications powered by Hadoop[EB/OL].[2011-06-15].http://wiki.apache.org/hadoop/PoweredBy.
  • 6Amazon.Amazon elastic compute cloud[EB/OL].[2011-06-15].http://aws.amazon.com/ec2.
  • 7Borthakur D.The hadoop distributed file system:architecture and design[EB/OL].[2011-06-15].http://hadoop.apache.org/hdfs/docs/current/hdfs _ design.html.
  • 8Zaharia M,Konwinski A,Joseph A D.Improving mapreduce performance in heterogeneous environments[C] //Proc of the 8th Usenix Symp on Operating Systems Design and Implementation.New York:ACM Press,2008:29-42.
  • 9Xie Jiong,Yin Shu,Ruan Xiao-jun,et al.Improving mapreduce performance through data placement in heterogeneous hadoop clusters[C] //IPDPS Workshops.Atlanta:IEEE Computer Society Press,2010:1-9.
  • 10Ghemawat S, Gobioff H, Leung ST. The Google file system. In: Proc. of the SOSP 2003. 2003.20-43. [doi: 10.1145/1165389. 945450].

共引文献178

同被引文献7

引证文献3

二级引证文献14

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部