面向微博热点事件的话题检测及表述方法研究

Research on topic detection and expression method for Weibo hot events

针对微博文本数据稀疏导致热点话题难以检测的问题,提出了一种基于IDLDA-ITextRank的话题检测模型。首先,通过引入微博时间序列特征和词频特征,构建了IDLDA话题文本聚类模型,利用该模型将同一话题的文本聚到一个文本集合TS;然后,通过采用编辑距离和字向量相结合的相似度计算方法,构建了ITextRank文本摘要和关键词抽取模型,对文本集合TS抽取摘要及其关键词;最后,利用词语互信息和左右信息熵将所抽取的关键词转换成关键主题短语,再将关键主题短语和摘要相结合对话题内容进行表述。通过实验表明,IDLDA模型相较于传统的BTM和LDA模型对话题文本的聚类效果更好,利用关键主题短语和摘要对微博的话题进行表述,比直接利用主题词进行话题表述具有更好的可理解性。

Aiming at the problem that Weibo text data sparseness is difficult to detect hot topics,this paper proposed a topic detection model based on IDLDA-ITextRank. Firstly,this paper constructed an IDLDA topic text clustering model by introducing the Weibo time series features and word frequency features,and used the model to cluster the text of the same topic into a text set( TS). Secondly,by using the similarity calculation method which combining editing distance and word vector,it constructed the ITextRank text summary and keyword extraction model to extract the summaries and keywords of the TS. Finally,it used the mutual information of words and left-right information entropy to convert the extracted keywords into key topic phrases,and combined the key topic phrases and summaries to express the topic content. Experiments show that IDLDA model has better clustering effect on topic text than the traditional BTM model and LDA model. The key topic phrases and summaries are better than keywords to express and understand the topics of Weibo.