基于改进隐式狄利克雷分布算法的新浪微博话题检测

Topic Detection from Sina Microblog Based on Improved Latent Dirichlet Allocation Algorithm

提出了一个基于改进主题模型的微博话题检测算法。由于传统的主题模型主要适用于传统媒体文字,对短文本的处理效果不是很好。针对微博文本特有的数据结构,在文本聚类时先加入转发特征以及评论特征权重预处理方法。在此基础上,改进了传统的隐含狄利克雷分布主题模型用来提取热点微博数据中的主题。实验证明,与传统相比方法,改进的主题模型解决了传统主题检测方法在应用于短文本时存在的高维稀疏问题。

This paper presents a topic detection algorithm from microblog based on improved topic model.Because the traditional topic model is mainly applied to traditional media text,the results of applied to short text is not effective.For the data structure of microblogging text,this paper puts forward the forwarding feature and the feature weight preprocessing methodapproach during the text clustering step.Besides,this paper improves the latent dirichlet allocation methodapproach and applied it to extract the topics of microblog.