基于马氏距离的异构网络异常大数据剔除方法

Abnormal Big Data Removal Method Based on Mahalanobis Distance in Heterogeneous Networks

传统异构网络异常大数据剔除方法存在数据维度较高、噪声较明显问题,导致异常数据剔除率偏低,且方法精度也不够理想。研究提出基于马氏距离的异构网络异常大数据剔除方法。利用改进马氏距离降维处理异构网络数据,分析数据之间相关性,提取网络数据主成分,生成具有较强抗噪性的高斯加权核函数。通过降维处理后的网络数据构建异常大数据信息流模型,利用固有模态将异常大数据信号分解成若干个窄带信号,通过特征点对的匹配实现异构网络异常大数据的高效剔除。实验结果表明,上述方法能够确保信号幅值大于噪声幅值,提升所提方法的异常大数据检测能力,在数据信噪比为-15dB时,剔除率可达100%,实验数据验证了所提方法具备高效的异构网络异常大数据剔除能力。

The traditional heterogeneous network abnormal big data elimination method has a low elimination rate and accuracy due to high data dimension and obvious noise. This paper reported a method to eliminate abnormal big data in heterogeneous networks based on Markov distance. Mahalanobis distance was improved to reduce the dimension and deal with heterogeneous network data. The relationship between the data was investigated in detail. The principal components of network data were extracted to generate Gaussian weighted kernel function with strong noise resistance. The abnormal big data information flow model was constructed to reduce the dimensionality of the processed network data. The natural mode was applied to decompose the abnormal big data signal into many narrow-band signals. According to the matching results of feature point pairs, the efficient elimination of abnormal big data in heterogeneous networks was achieved. The experimental results show that this method can detect abnormal big data and has a high rejection rate(signal-to-noise ratio-15 dB,rejection rate up to 100%).