分布式存储模式下的数据错误检测方法综述

Review on Data Error Detection Methods in Distributed Storage Mode

数据错误检测是数据质量保证的重要环节,直接关系到数据全生命周期分析结果的可信度。随着云边端数据中心架构应用领域及范围的逐渐扩大,以及网络节点存储计算能力的提升,数据分布式本地存储日益普遍,传统数据集中式存储模式下的数据错误检测方法难以适应数据分布式存储模式。基于此,开展分布式存储模式下的数据错误检测方法综述,在数据错误检测问题描述与分类基础上,从技术原理、模型方法、主要进展等角度,对基于传统分布式学习的数据错误检测方法、基于联邦学习框架的数据错误检测方法进行总结分析,比较了二者之间的区别及联系,并展望提出领域相关潜在研究机会及关注问题,为开展分布式存储模式下的数据错误检测及相关研究提供借鉴和参考。

Data error detection is an important part of data quality assurance,which is directly related to the reliability of data lifecycle analysis results.With the gradual expansion of the application field and scope of the cloud-edge data center architecture,and the improvement of the storage and computing power of network nodes,distributed local storage of data is becoming more and more common.The data error detection method under the traditional data centralized storage mode is difficult to adapt to the data distributed storage mode.Based on this,this paper carries out a survey of data error detection methods in distributed storage mode.On the basis of the description and classification of data error detection problems,this paper summarizes and analyzes data error detection methods based on traditional distributed learning and data error detection methods on account of federated learning framework from the perspectives of technical principles,model methods and main progress.The differences and connections between them are compared,and the potential research opportunities and concerns in the field are prospected.This paper provides reference for data error detection and related research in distributed storage mode.