基于自编码网络特征降维的舆情异常检测技术研究

Research on public opinion anomaly detection technology based on AE network feature dimension reduction

传统的网络舆情监控通常采用人工或简易图片识别软件辅以自然语言理解算法进行,针对这些舆情异常检测识别方法中存在的效率较低、准确率不足及难以处理海量舆情信息等问题,文中提出一种基于自编码网络特征降维的舆情异常检测方法。该方法采用多层受限玻尔兹曼机(RBM)将原始数据中的高维、非线性数据映射至相应的低维空间。同时,还通过重构与权重微调算法将低维空间中的数据在高维空间进行最优化的高维表示,进而获得原始数据的最优低维表示,以实现数据的特征降维。最后,利用SVM算法对降维后的数据进行分类,实现对网络舆情的异常检测。实验结果表明,所提方法的网络舆情异常检测准确率以及检测率均可达到97%以上,充分验证了该方法的有效性。

The traditional network public opinion monitoring is usually carried out by means of artificial or simple image recognition software supplemented by natural language understanding algorithm.A public opinion anomaly detection method based on AE(AutoEncoder)network feature dimension reduction is proposed to improve low efficiency,insufficient accuracy and difficulty in handling massive public opinion information in these public opinion anomaly detection and recognition methods.In this method,multilevel restricted Boltzmann machine(RBM)is used to map the high-dimensional and nonlinear data in the original data to the corresponding low dimensional space.The high-dimensional representation of the data in the low-dimensional space in the high-dimensional space is conducted by means of the reconstruction and weight fine-tuning algorithm to obtain the optimal low-dimensional representation of the original data,so as to realize the feature dimension reduction of the data.The SVM algorithm is used to classify the data after dimensionality reduction,so as to realize the anomaly detection of network public opinion.The experimental results show that the accuracy and detection rate of the proposed method can reach more than 97%,which fully verifies the effectiveness of this method.