抑制背景噪声的LDA子话题挖掘算法

LDA Subtopic Detection Algorithm with Background Noise Restraint

专题文章集合是一些拥有相似背景知识的文章集合.为了更好地从专题文章集合内部的复杂信息关联中高效挖掘子话题信息,文中提出了抑制背景噪声的线性判别分析(LDA)子话题挖掘算法BLDA,通过预先抽取专题文档集合的共同背景知识、在迭代过程中重设关键词的产生等方式提高子话题抽取的准确程度.在微信公众账号文章上的系列实验证明,BLDA算法针对有共同背景的专题文章集合的聚类结果显著优于传统的LDA算法,其中主题召回率提高了170%,Purity聚类指标提高了143%,NMI聚类指标提高了160%.

Special article set is a collection of articles with common background knowledge. In order to more effec-tively detect the subtopics form special article set with complex information correlation, an LDA subtopic detection algorithm with background noise restraintnamed BLDA is proposed, which improves the precision of subtopic detec-tion from article set by firstly extracting the common background knowledge and then reproducing the keywords in each iteration step. By a series of experiments on a set of WeChat documents from public accounts, it is proved that the detection results obtained by BLDA are much better than those obtained by LDA, with a topic recall rate incre-ment of about 170% , a Purity index increment of 143% and a NMI index increment of 160%.