微博热词抽取及话题发现研究

Micro-blogging Hot Words Extraction and Topic Detection

旨在实践一种方法,能从大量散乱的微博语料中快速抽取热点词语并从中形成话题。首先参考文本串的词频、内部聚合度、粘联度这三个统计特征判断文本串是否成词,直接从样本语料中抽取出高频词。然后根据这些高频词在不同时间窗口的出现频率筛选出实时热词,最后利用词共现确定热词间的关联度,将热词聚类形成热点话题。实验证明,该算法简单易行,在话题发现中取得了良好的效果。

This paper aims to extract valuable information from massive fragmented content and feed back to the user in a concise form. Firstly, considering three statistical characteristics： word frequency of text string, internal degree of coupling, the external degree of flexi- bility, we extract high-frequency words from micro-blog corpus, then filter outreal-time hot words according to the frequency of occur- rence of these high-frequency words in different time windows, and finally use the word co-occurrence to determine the hot words correla- tion to get a hot topic. Experimental results show that the algorithm is simple and available, and achieved good results on the topic detec- tion.