期刊文献+

基于校验编码备份的分布存储方案

Distributed storage solution based on parity coding
下载PDF
导出
摘要 传统的云计算存储系统为保障可用性,一般使用镜像冗余备份而产生大量冗余备份数据,影响了存储数据空间的利用效率。针对此情况,为减少备份数据对存储空间的占用,提出一种存储方案。放弃了镜像冗余备份,引入校验编码的方式进行备份,以减少备份数据;同时采用了冲突跳转的机制对备份进行验证,在保证备份数据有效性的前提下减少备份数量。通过模拟程序运行结果与主流云存储方案的对比表明,所提存储方案在保证数据可靠性的同时,显著地降低了分布存储对磁盘空间的占用。 To guarantee reliability, traditional cloud storage solutions generally backup data through mirror redundancy, which influences the usage efficiency of storage data space. A storage solution was proposed to reduce the usage of storage data space for redundancy-backup data. The solution introduced: 1) the parity coding backup instead of mirror backup, which reduced the size of backup data; 2) the conflict-jump mechanism to confirm the backup data, which guaranteed rehability while number of backup data copies was reduced. The contrast between running result of simulation program and performance of mainstream cloud storage solutions shows that, by using the proposed solution, the usage of storage space for distributed storage is significantly reduced while the reliability gets guaranteed.
作者 陈冬晓 王鹏
出处 《计算机应用》 CSCD 北大核心 2013年第1期211-214,218,共5页 journal of Computer Applications
基金 广东省科技厅高新技术产业化科技攻关项目(2011B010200007) 成都市科技局创新发展战略研究项目(软科学)(11RKYB016ZF)
关键词 云存储 一致性哈希 HADOOP分布式文件系统 数据备份 数据恢复 cloud storage consistent hash Hadoop Distributed File System (HDFS) data backup data recovery
  • 相关文献

参考文献13

  • 1Storage Networking Industry Association (SNIA). Cloud storage ref- erence model: Version O. 3 rev [ EB/OL]. [2012-03-15]. http:// www. snia. org/sites/default/files/CloudStorageReferenceModelV03. pdf.
  • 2CALLAGHAN B. NFS over RDMA [ EB/OL]. [2012-04-05]. ht- tp://static, usenix, org/events/fastO2/wips/callagcal, pdf.
  • 3BORTHAKUR D. HDFS architecture guide [ EB/OL]. [2012-03- 18]. http://hadoop, apache, org/common/docs/r0.20.0/hdfs_de- sign. pal.
  • 4SHVACHKO K, KUANG H, RADIA S, et al. The Hadoop distrib- uted file system [ C]// Proceedings of the 2010 IEEE 26th Symposi- um on Mass Storage Systems and Technologies. Washington, DC: IEEE Computer Society, 2010:1 - 10.
  • 5许春玲,张广泉.分布式文件系统Hadoop HDFS与传统文件系统Linux FS的比较与分析[J].苏州大学学报(工科版),2010,30(4):5-9. 被引量:19
  • 6WHITE T. HDFS reliability [ EB/OL]. [ 2012- 04- 20}. http:// www. eloudera, com/wp-eontent/uploads/2010/03/HDFS_Reliabil- ity. pdf.
  • 7DECANDIA G, HASTORUN D, JAMPANI M, et al. Dynamo: Amazon's highly available key-value store [ C]//SOSP'07: Proceed- ings of the 21st ACM Symposium on Operating Systems Principles. New York: ACM Press, 2007:205 -220.
  • 8Amazon. corn Inc. Amazon simple storage service (Amazon S3) [ EB/OL]. [ 2012-03-15]. http://aws, amazon, corn/s3.
  • 9LEWIN D. Consistent hashing and random trees: algorithms for cac- hing in distributed networks [ D]. Cambridge, Massachusetts: Mas- sachusetts Institute of Technology, 1998.
  • 10PAl V S, ARON M, BANGA G, et al. Locaity-aware request distribu- tion in cluster-based network servers [ C]// Proceedings of the 8th In- ternational Coxfference on Architectural Support for Programming Lan-guages and Operating Systems. New York: ACM Press, 1998:205 -216.

二级参考文献17

  • 1John Howard,Michael Kazar,Sherri Menees,et al.Scale and performance in a distributed file system[J].ACM Transactions on Computer Systems,1988,6(1).
  • 2Luiz A Barroso,Jeffrey Dean,Urs H¨olzle.Web search for a planet:the Google cluster architecture[J].IEEE Micro,2003,23(2).
  • 3Storage Networking Industry Association (SNIA).Cloud storage reference model:Version 0.3 rev[S/OL].[2011-07-18].http://www.snia.org/sites/default/files/CloudStorageReferenceModelV03.pdf.
  • 4Amazon.com Inc.Amazon simple storage service (Amazon S3)[EB/OL].[2011-07-18].http://aws.amazon.com/s3.
  • 5BORTHAKU D.The Hadoop distributed file system:Architecture and design[EB/OL].[2011-07-05].http://hadoop.apache.org/common/docs/r0.18.0/hdfs_design.pdf.
  • 6Hbase Development Team.HBase:Bigtable-like structured storage for Hadoop HDFS[EB/OL].[2011-06-12].http://wiki.apache.org/hadoop/Hbase.
  • 7盛大网络.盛大云——云硬盘[EB/OL].[2011-05-18].http://www.grandcloud.cn/index.php?m=content&c=index&a=lists&catid=29.
  • 8盛大网络.盛大云——云存储[EB/OL].[2011-05-28].http://www.grandcloud.cn/index.php? m =content&c =index&a =lists&catid =18.
  • 9ZENG WENYING,ZHAO YUELONG,OU KAIRI,et al.Research on cloud storage architecture and key technologies[C]//ICIS '09:Proceedings of the 2nd International Conference on Interaction Sciences:Information Technology,Culture and Human.New York:ACM,2009:1044-1048.
  • 10LITWIN W,NEIMAT M-A,SCHNEIDER D A.LH *-A scalable,distributed data structure[J].ACM Transactions on Database Systems,1996,21(4):480-525.

共引文献44

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部