基于LDA模型的新闻话题分类研究

Study on Classification of News Topic Based on LDA Model

针对部分网站中新闻话题没有分类或者分类不清等问题，将LDA模型应用到新闻话题的分类中。首先对新闻数据集进行LDA主题建模，根据贝叶斯标准方法选择最佳主题数，采用Gibbs抽样间接计算出模型参数，得到数据集的主题概率分布；然后根据JS距离计算文档之间的语义相似度，得到相似度矩阵；最后利用增量文本聚类算法对新闻文档聚类，将新闻话题分成若干个不同结构的子话题。实验结果显示表明该方法能有效地实现对新闻话题的划分。

The LDA model is applied to the classification of news topic on the website because of its no classification or unclear classification. Firstly, news dataset is modeled by LDA modeling, the optimal number of topic is chosen according to Bias standard method, and get the topic probability distribution of dataset by using Gibbs sampling to calculate the model parameters;and then similarity matrix is obtained based on the semantic similarity between documents by computing JS distance;finally, the incremen-tal clustering algorithm is used to cluster news document, and the topic is divided into a number of different structure of the sub topic. The experimental results show that this method can realize the division of news topic effectively.