文本聚类算法在舆情监控中的应用分析

Applied research of text clustering algorithm in network monitoring public opinion

为满足网络舆情监控系统中话题发现的需要,并克服经典single-pass算法在处理网络文本聚类时受输入顺序及精度较低的不足,本文对single-pass聚类算法进行改进,通过采用average-1ink策略及引入"代"的思想分批聚类,在借鉴single-pass聚类方法简单高效的同时,又克服了其缺点,兼顾了网络话题发现的实时性和准确性,通过实验分析改进后的single-pass算法比single-pass算法在漏检率、误检率和耗时方面都有很大改观。实验证明改进的算法在提高话题发现准确度上的有效性和实用性。

To meet the needs of topic detection for monitoring the public opinion on internet,this paper proposed an incremental clustering algorithm to improve the two main disadvantages of single-pass algorithm,that was,being easily effected by the order of inputs and low precision. In this paper, the single-pass clustering algorithm the average-link strategy and the introduction of the idea of ＂generation＂ in batches clustering inherited the simple principle from single-pass to ensure clustering internet texts in real time and overcame. Tough the experimental analysis of the improved single-pass algorithm than the single-pass algorithm in fhe miss rate, error rate and time consuming aspects has greatly improved. The experimental results show the improved algorithm in improving the topic detection accuracy on the validity and practicality.