期刊文献+

一种改进的云存储系统容错机制 被引量:3

Improved fault-tolerance mechanism in cloud storage system
下载PDF
导出
摘要 在云存储平台下,提出了一种基于访问统计的自适应容错机制SFMAF,该机制通过近似最近最少使用算法维护一张文件访问频率表来自适应调整容错方式。SFMAF对于常读取的文件采用副本冗余机制;对于不常读取的文件采用Reed-Solomon(RS)纠删码容错机制。实验结果表明,SFMAF相对于副本冗余机制,在CPU和内存使用率可接受范围内的增加上,减少了系统内部数据的传输流量,即减少了系统存储空间。 This paper presented a self-adaptive fauh-tolerance mechanism based access frequency (SFMAF) in cloud storage platform. This mechanism was implemented by maintaining a file access frequency table using approximate least recently used algorithm. If the file was read frequently, it used replica redundancy mechanism. If the file was seldom read, it used RS erasure code mechanism. Experimental results show that, comparing to replica redundancy mechanism, SFMAF increases CPU and memory utilization within the acceptable range, and reduces the data transmission flow within the system, that reduces the storage space in system.
出处 《计算机应用研究》 CSCD 北大核心 2013年第12期3724-3728,共5页 Application Research of Computers
基金 国家教育部互联网应用创新开放平台示范基地项目(教技发中心函[2012]120号) 国家自然科学基金资助项目(61370178) 国家教育部-中国移动科研基金资助项目(MCM20121051)
关键词 云存储 纠删码 文件访问频率表 自适应 容错机制 cloud storage erasure code file access frequency table self-adaptive fault-tolerant mechanism
  • 相关文献

参考文献11

  • 1张建勋,古志民,郑超.云计算研究进展综述[J].计算机应用研究,2010,27(2):429-433. 被引量:586
  • 2陈康,郑纬民.云计算:系统实例与研究现状[J].软件学报,2009,20(5):1337-1348. 被引量:1310
  • 3ZAWODNY J.YahooJ launches World's largest HadoopProduction ap-plication[EB/OL].(2008-02-19).http://developer.yahoo,com/blogs/hadoop/posts/2008/02/yahoo-worlds-largest-production-ha-doop/.
  • 4GIBSON G,METER R.Network attached storage architecture[J].Communications of the ACM,2000,43(11):37-45.
  • 5马玮骏,吴海佳,刘鹏.MassCloud云存储系统构架及可靠性机制[J].河海大学学报(自然科学版),2011,39(3):348-354. 被引量:16
  • 6Object storage developer documentation swift[EB/OL].(2012-03-12).http://swift,openstack.org/.
  • 7RIZZO L.On the feasibility of software FEC[EB/OL].(1997-01-31).http://teamfrednet-mockups.googlecode.com/files/Rizzo-Feas-Soft-FEC-97.pdf.
  • 8REED I S,SOLOMON G.Polynomial codes over certain finite fields[J].Journal of the Society for Industrial and Applied Mathe-matics,I960,8(2):30Q-304.
  • 9罗象宏,舒继武.存储系统中的纠删码研究综述[J].计算机研究与发展,2012,49(1):1-11. 被引量:92
  • 10PLANK J S.A tutorial on reed-solomon coding for fault-tolerance inRAID-like systems[J].Software Practice and Experience,1997,27(9):995-1012.

二级参考文献122

  • 1ZHOUKe HUANGYongfeng ZHANGJiangling FENGDan WANGFang.Disk Tree——A Case of Parallel Storage Architecture to Improve Performance in Random Access Pattern[J].Chinese Journal of Electronics,2005,14(1):39-44. 被引量:2
  • 2韩作生,袁东风.RS码频域编译码的计算机模拟[J].通信学报,1994,15(6):104-112. 被引量:11
  • 3田荣华,卢显良,侯孟书,王晓斌.P2P分布式存储系统[J].计算机科学,2007,34(6):47-48. 被引量:8
  • 4Sims K. IBM introduces ready-to-use cloud computing collaboration services get clients started with cloud computing. 2007. http://www-03.ibm.com/press/us/en/pressrelease/22613.wss
  • 5Boss G, Malladi P, Quan D, Legregni L, Hall H. Cloud computing. IBM White Paper, 2007. http://download.boulder.ibm.com/ ibmdl/pub/software/dw/wes/hipods/Cloud_computing_wp_final_8Oct.pdf
  • 6Zhang YX, Zhou YZ. 4VP+: A novel meta OS approach for streaming programs in ubiquitous computing. In: Proc. of IEEE the 21st Int'l Conf. on Advanced Information Networking and Applications (AINA 2007). Los Alamitos: IEEE Computer Society, 2007. 394-403.
  • 7Zhang YX, Zhou YZ. Transparent Computing: A new paradigm for pervasive computing. In: Ma JH, Jin H, Yang LT, Tsai JJP, eds. Proc. of the 3rd Int'l Conf. on Ubiquitous Intelligence and Computing (UIC 2006). Berlin, Heidelberg: Springer-Verlag, 2006. 1-11.
  • 8Barroso LA, Dean J, Holzle U. Web search for a planet: The Google cluster architecture. IEEE Micro, 2003,23(2):22-28.
  • 9Brin S, Page L. The anatomy of a large-scale hypertextual Web search engine. Computer Networks, 1998,30(1-7): 107-117.
  • 10Ghemawat S, Gobioff H, Leung ST. The Google file system. In: Proc. of the 19th ACM Symp. on Operating Systems Principles. New York: ACM Press, 2003.29-43.

共引文献1908

同被引文献27

  • 1董新华,李瑞轩,周湾湾,王聪,薛正元,廖东杰.Hadoop系统性能优化与功能增强综述[J].计算机研究与发展,2013,50(S2):1-15. 被引量:69
  • 2王鹏,王新梅.LDPC码的快速编码研究[J].西安电子科技大学学报,2004,31(6):934-938. 被引量:20
  • 3Sage A Weil,Scott A Brandt,Ethan L Miller,et al.Ceph:A Scalable,High-Performance Distributed File System[C].Proceedings of the 7th symposium on Operating systems design and implementation,2006.
  • 4Plank J S,Simmerman S,Schuman C D.Jerasure:A library in C/C++facilitating erasure coding for storage applications-Version 1.2[Z].University of Tennessee,Tech.Rep.CS-08-627,2008,23.
  • 5Panian Z. A new data management challenge: How to handle big datal /Proceedings of the International Conference on Humanities. Geography and Economics. Dubai , UAE. 2013: 47-51.
  • 6Rousseau R. A view on big data and its relation to informetrics. ChineseJournal of Library and Information Science. 2012. 5(3): 12-26.
  • 7Zhu Yun-Feng , Lee P P C. Hu Yu-Chong , et al. On the speedup of single-disk failure recovery in XOR-coded storage systems: Theory and practice//Proceedings of the 28th IEEE Conference on Massive. London. UK. 2012: 106-114.
  • 8Cui Ii-Feng , Zhang Yong , Li Chao. Xing Chun-Xiao. A packaging approach for massive amounts of small geospatial files with HDFS//Proceedings of the Web-Age Information Management. Beijing. China. 2012: 210-215.
  • 9Dong Bo, Zheng Qing-Hua. Tian Feng , et al. Performance models and dynamic characteristics analysis for HDFS write and read operations: A systematic view.Journal of Systems and Software. 2014. 93: 132-151.
  • 10Wang Yong-Gang , Wang Sheng. Research and implementation on spatial data storage and operation based on hadoop platform//Proceedings of the 2010 2nd IITA International Conference on Geoscience and Remote Sensing. Qingdao. China. 2010: 275-278.

引证文献3

二级引证文献18

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部