云计算下相关性缺失大数据分块填补仿真

Simulation of Large Data Block Filling for Relativity Missing in Cloud Computing

传统数据填补手段填补规模受限,存在运行不稳定、内存占比较大以及填补精度较低等缺点,为此提出一种云计算下相关性缺失大数据分块填补。根据数据填补原理,可通过较小的区间代替缺失数据,计算大数据集信息熵与指标之间的相关性系数,将数据集填充于原始大数据中,计算新得到的数据集信息熵,利用新旧信息熵的相似性关系扩大区间范围。随后对相关性缺失大数据做分块处理,分成已知分块和未知分块,已知分块可以直接对其进行填补,未知分块需要利用基于稀疏性的K-means算法约束目标函数中变量权重,并划分其聚类结果获得未知分块数据集,最后利用宿主法实现填补。仿真结果证明,所提方法相比其它方法,精准度较高、填补效果良好且运行稳定。

In traditional data filling methods, the scale of filling is limited, leading to unstable operation, large memory and low accuracy. Therefore, this article focuses on a method to separately fill with missing big data of correlation in cloud computing. According to the data filling principle, the missing data could be replaced by small intervals. It was necessary to calculate the correlation coefficient between the information entropy of big data set and the index, and fill the data set in the original big data. And then, the information entropy of new data set was calculated. The similarity relationship between the old information entropy and new information entropy was used to expand the range of interval. In addition, the missing big data of correlation was divided into known blocks and unknown blocks. It was able to fill known blocks directly, but the unknown blocks needed to use the K-means algorithm based on sparsity to constrain the variable weights in objective function. Moreover, the clustering results were divided to get unknown blocked data sets. Finally, the host method was adopted to realize the data filling. Simulation results show that the proposed method has higher accuracy, better filling effect and stable operation.