基于词频均值波动和概率语言模型的短文本热点主题探测研究

Short Texts’ Hot Topics Detection: Based on Word Frequency Mean Fluctuation and Probabilistic Language Model

[目的/意义]针对短文本词语特征向量稀疏的问题,提出利用突然爆发现象和词语共现概率现象进行热点主题探测研究的新视角。[方法/过程]以此为目标建立了一个分析框架,以词频均值波动模型发现短文本中的热点词,以概率语言模型发现主题词,再对二者结果集合进行相似度计算实现热点主题的探测与展示。[结果/结论]通过对较高热度噪声词的过滤以及热点事件词的发现实现热点主题的探测。通过对比谷歌趋势的结果,准确率达到82.67%,证明模型有效。本研究对短文本热点主题探测的理论和实践研究具有一定参考价值。

[Purpose/Significance]Aiming at the problem of sparse feature vectors of short texts, the research proposes a new research perspective for hot topics detection by using sudden outbreak phenomenon and word co-occurrence probability phenomenon.[Method/Process]We set up an analytical framework to find hot words in short texts by word frequency mean fluctuation model and to detect topic words by probabilistic language model, and then calculated the similarity between their results to display the hot topics.[Result/Conclusion]The method detects hot topics by filtering high thermal noise words and discovering hot event words. By comparing the results of Google trend, the accuracy rate is 82.67%, which proves that the model is effective. The research is helpful for actual business practice or theory study of short texts’ hot topics detection.