基于Biterm主题模型的新闻线索生成方法

Generating News Clues with Biterm Topic Model

【目的】为解决生成新闻线索时抽取新闻主题及度量子事件相关性困难的问题,通过动态滑动窗口的方法改进主题模型,提高长文本和短文本新闻主题抽取的质量,并基于抽取出的新闻主题,提出面向新闻事件的新闻线索生成方法。【方法】在主题模型IBTM(Incremental Biterm Topic Model)的基础上,通过动态滑动窗口减小二元词组的提取范围,提出既适合在长文本新闻也适合在短文本新闻上抽取主题的News-IBTM模型,进而基于该模型从新闻数据中抽取主题分布和主题-词分布、推断文档-主题分布,再利用JS散度来度量文档-主题分布的差异,从而生成新闻线索。【结果】在人民网新闻和微博新闻数据上的实验结果表明,无论是长文本新闻还是短文本新闻,News-IBTM在困惑度、准确率及效率上都优于现有的经典主题模型。【局限】News-IBTM以及其他新闻线索生成方法的准确率都不高,还可以进一步提升。【结论】本文方法适合应对长文本和短文本新闻主题抽取的质量问题,并能从新闻事件中获取新闻线索。

[Objective]This paper modifies the topic model to improve the quality of extracted news clues.[Methods]We constructed a News-IBTM model based on IBTM(Incremental Biterm Topic Model)with dynamic sliding window,which reduced the extraction scope of binary phrases.Then,we used this model to extract topics and topic-word distributions from news,and inferred the document-topic distributions.Finally,we used the JS(Jensen-Shannon)divergence to measure the difference between document-topic distributions and generate news clues.[Results]We examined our News-IBTM model with news from People’s Daily Online and Weibo.The proposed model outperformed existing ones in perplexity,accuracy and efficiency.[Limitations]The accuracy of News-IBTM algorithm needs to be further improved.[Conclusions]The proposed method could effectively extract quality news topics and clues.