基于信息交叉熵的脏数据识别与修正技术研究

Research on Dirty Data Recognition and Correction Technology Based on Information Cross Entropy

网络中存在异常数据是不可避免的,为避免脏数据影响网络正常运行,提出一种基于信息交叉熵的脏数据识别与修正技术研究方法。采用基于小波的方法对网络数据进行预处理,并结合信息熵与相对熵构建交叉熵,实现对网络数据的约简;采用熵目标函数,设置一个阈值,实现对系统中的脏物进行检测,并采用RBF神经网络模型进行重建,实现对脏物的校正。实验结果表明,针对不同类型脏数据,所提技术脏数据识别能力好、识别精度高以及信息熵丢失少。

The existence of abnormal data in the network is inevitable.In order to avoid the influence of dirty data on the normal operation of the network,a research method of dirty data identification and correction technology based on information cross entropy is proposed.The method based on wavelet is used to preprocess the network data,and the information entropy and relative entropy are combined to construct cross entropy to achieve the reduction of network data.Using entropy objective function,setting a threshold to detect dirt in the system,and using RBF neural network model for reconstruction to achieve correction of dirt.Experimental results show that for different types of dirty data,the proposed technology has good ability to identify dirty data,high accuracy and less loss of information entropy.