基于分数阶Fourier变换的云存储系统重复数据删除算法

Duplicate Data Remove Algorithm of Cloud Storage System Based on Fractional Fourier Transform

云存储系统的重复数据作为大量冗余数据的一种,对其有效及时地删除能保证云存储系统的稳定与运行。由于云存储系统中的干扰数据较多,信噪比较低,传统的重删算法会在分数阶Fourier域出现伪峰峰值,不能有效地对重复数据进行检测滤波和删除处理,因此提出一种改进的基于分数阶Fourier变换累积量检测的云存储系统重复数据删除算法。首先分析云存储系统重复数据删除机制体系架构,定义数据存储点的适应度函数,得到云存储节点的系统子集随机概率分布;采用经验约束函数对存储节点中的校验数据块分存,通过分数阶Fourier变换对云存储系统中的幅度调制分量进行残差信号滤波预处理。采用4阶累积量切片后置算子,把每个文件分为若干个块,针对每个文件块进行重删,进行重复数据检测后置滤波处理,实现存储资源上的重复数据检测及其删除。仿真实验表明,该算法能提高集群云存储系统计算资源的利用率,重复数据准确删除率较高,有效避免了数据信息流的干扰特征造成的误删和漏删,性能优越。

Duplicate data of cloud storage system is taken as one of a large amount of redundant data,and the effective and timely remove can guarantee the stability and operation of cloud storage system.Because of the interference of data,the SNR is low,the traditional method has false peaks in the fractional Fourier domain,and it cannot effectively detect and remove the duplicate data.An improved duplicate data remove algorithm of cloud storage system was proposed based on fractional Fourier transform cumulant detection.Firstly,the delete system architecture for cloud storage system was taken,the fitness function of data storage point was defined,and system subset random probability distribution function of the cloud storage node was gotten.The constraint function was used for blocking the calibration data of storage nodes,the detection of duplicate data removing processing was taken,and the fractional Fourier transform was used to preprocess the residual signal filtering in cloud storage system.The 4 order cumulanted slice post operator was used to divide each file into blocks.To delete each file block,duplicated data detection post filtering was obtained,and data storage resource detection and deletion were realized.Simulation results show that this algorithm can improve the utilization efficiency of cluster cloud storage system resource,and duplicate data can be accurately removed with higher rate.It can effectively avoid the error removing caused by interference and leakage removing,and it has superior performance.