基于堆叠式分布式文件系统的端到端校验

End-to-end Data Integrity for Stacked Distributed File System

端到端校验是一种有效的数据完整性检测手段,可为分布式存储系统提供基本的可靠性保证。Glusterfs是一种常用的堆叠式分布式文件系统,但缺乏有效的数据完整性检测机制,存在用户数据遭受破坏而无法被发现的风险,即返回错误数据给用户。这种风险在某些情况还会扩散,造成多副本或灾备、双活情况下的数据丢失。针对这一问题,该文提出了一种高性价比的基于Glusterfs的端到端校验方案(命名为Glusterfs-E2E),可以有效解决Glusterfs文件系统中存在的数据完整性风险。该方案不但可以提供全路径的保护,具备2%~8%的高性能开销,而且还可以提供软件故障的定位功能。

End-to-end checksum is an effective means of data integrity detection,which can provide basic reliability guarantee for the distributed storage systems.Glusterfs is a popular stacked distributed file system,but it lacks an effective data integrity detection mechanism.User data storage in the Glusterfs have a risk of being damaged and not being discovered.Moreover,this kind of risk can spread in some cases,causing data loss even with the protection of multiple copies or disaster recovery.This paper proposes a cost-effective Glusterfsbased end-to-end checksum scheme called Glusterfs-E2 E,which can effectively solve the data integrity risk of Glusterfs.The proposed solution can not only provide full path protection,2%to 8%performance overhead,but also can locate software bugs.