基于随机矩阵分解的大数据无向压缩算法设计

Design of Big Data Undirected Compression Algorithm Based on Random Matrix Decomposition

由于已有的大数据压缩算法均是按照一定方向实现,导致冗余数据无法完全滤除,数据压缩耗时较长且网络能耗较大。为解决上述问题,设计一种基于随机矩阵分解的大数据无向压缩算法。构建随机矩阵分解模型,引入用户相邻数据计算隐含可靠大数据,消除冗余信息;运用归约技术预处理数据,利用不同变量的线性组合顶替初始变量,标准化处理大数据样本降低压缩难度;构建数据压缩评估指标,采用无向旋转门算法,期望误差和真实解压缩误差作为负反馈,动态调节压缩误差,连续迭代直到误差降至理想范围,完成大数据无向压缩算法的设计。仿真结果表明,所提算法压缩耗时短,有效提升了大数据压缩比,减少了网络能耗,为大数据的管理与应用提供参考借鉴。

Because the existing algorithms are implemented according to a certain direction,redundant data can't be completely filtered out,and the data compression is time-consuming.Therefore,this paper presented an algorithm of undirected compression for big data based on random matrix decomposition.Firstly,a random matrix decomposition model was built.And then,users' adjacent data were introduced to calculate the hidden reliable big data,thus eliminating redundant information.Secondly,reduction technology was applied to preprocess the data,and the linear combination of different variables was used to replace the initial variables.After that,the big data samples were standardized to reduce the difficulty of compression.Thirdly,the data compression evaluation indexes were constructed.Taking the expected error and the real decompression error as negative feedback,the undirected revolving door algorithm was adopted to dynamically adjust the compression error and continuously iterate over them until the error was reduced to an ideal proportion.Finally,the design of the algorithm was finished.Simulation results show that the proposed algorithm has short compression time,effectively improves the big data compression ratio,reduces the network energy consumption,and provides references for the management and application of big data.