一种网络编码分布式存储系统中的数据更新策略

An Update Scheme for Network-coding-based Distributed Storage System

网络编码作为一种具有高可靠性和低冗余等优点的存储方案而被引入到分布式存储系统中,但是其数据更新过程的开销却很大.一些更新方案为了减少I/O开销而引入Log-based存储机制,而没有考虑优化网络带宽使用量.本文中,通过分析多个数据集,发现了大部分更新数据块存在着多次更新的情况,针对这一现象,我们提出一种更新方案CUM,利用网络编码的线性特性和更新节点的缓存区域,将更新块和校验块的信息缓存起来,当数据块多次更新时则可以直接在缓存中读取信息来计算新的校验块,从而减少了到存储节点上读取数据的带宽使用量.本文在NCFS的基础上建立缓存更新模型,并在真实的数据集上对CUM进行测试.实验结果显示CUM相比当前广泛使用的Differential Update Model(DUM)在更新性能上有很大改进,带宽使用量减少最多达总量的30%,更新时间最多减少24.57%,更新速度最多提高1.326倍.

Due to the advantages of high reliability and low redundancy in data storage, network coding is introduced into the distributed storage system as one of the storage solution, but the overhead of its update process is high. Some update scheme use log-based storage mechanism just to reduce I/O overhead without considering optimizing the usage of network bandwidth. In this paper, we found that many data blocks are updated for many times after analyzing multiple data sets. We propose an update scheme CUM,using the linear characteristic of network coding and adding a cache region in update node to storage the information of data block and parity block. When the data block update again, the information can be read from the cache region to compute the new parity block,which re- duces the amount of bandwidth usage to read data from storage nodes. We also run the cache-based update model on the basis of the NCFS and evaluate the performance of CUM on real data sets. The results of the experiment show that CUM reduces the total bandwidth usage and update time up to 30% and 24.57% respectively and speed up the update process by a factor of up to 1.326 compare to Differential Update Model （DUM）.