一种基于wirehair码的高可靠分布式存储方案的研究与实现

Research and Implementation of a Highly Reliable Distributed Storage Scheme Based on Wirehair Code

在分布式存储中,每一个分布式存储节点的数据可用性都是极其重要的。因此,必须加入相应的冗余措施来保障数据的可用性。而传统基于复制的冗余策略会随着数据可用性的提高而极大增加冗余开销。同时,随着分布式存储节点个数的增加,部分节点遭受物理攻击、窃取的可能性也相应增大,极大增加了对于敏感数据的安全威胁。文章在对Raptor码深入研究的基础上, 提出一种基于Raptor码的分布式存储方案。且通过大量实验测试,设计并实现了一种基于wirehair码(RaptorQ的开源实现)的分布式存储系统。该系统不仅能以较小的冗余实现较高的数据可用性,且通过编码过程增加了原始数据的安全性。

In the distributed storage, the data availability of each distributed storage node is very important. Therefore, the corresponding redundance measures must be taken to ensure the data availability. However, the redundance cost of the traditional redundance strategy based on duplication will greatly increase as data availability improves. Meanwhile, with the increasing of the number of distributed storage nodes, the possibility that some nodes suffer physical attack and theft also increases, which greatly increases the security threat to sensitive data. On the basis of deep research on Raptor code, this paper presents a distributed storage scheme based on Raptor code. And through a lot of experimental tests, a distributed storage system based on wirehair code （open source implementation of RaptorQ） is designed and implemented. The system can not only achieve higher data availability with smallerredundance, but also increase the security of the original data through the coding process.