基于文档主题结构和词图迭代的关键词抽取方法研究

Extracting Keywords Based on Topic Structure and Word Diagram Iteration

【目的】利用主题模型得到文本主题信息,将主题信息融入TextRank模型对其进行改进,提升文本关键词自动抽取的准确率和召回率。【方法】利用LDA对文档集进行主题建模,得到候选关键词的主题词分布和文档主题分布;结合候选关键词主题分布特征计算节点权重,加权文档–主题概率分布和主题–词概率分布特征作为节点的随机跳转概率;构建新的转移矩阵进行词图迭代,得到改进后的TextRank模型。【结果】采集南方周末网站1 559篇新闻文章进行实验。结果表明,本文模型的关键词抽取效果明显优于原始TextRank和TF-IDF模型;当抽取关键词个数为3时,本模型准确率比原始TextRank模型的准确率提升4.7%,比TF-IDF提升6.5%。【局限】融合算法的计算复杂度增加。【结论】融合主题信息的TextRank算法能够使关键词自动抽取效果更加显著。

[Objective]This paper integrates the topic information to the TextRank model,aiming to improve the precision and recall of automatic keyword extraction.[Methods]First,we used the LDA to create a model for document topics,and obtained the topic distribution of the candidate keywords.Then,we calculated the node weights with the topic-word probability distribution features.Third,we weighted the probability distributions of document-topic and topic-word characteristics as the node’s random jump probability.Finally,we constructed a new transition matrix for word graph iteration to improve the TextRank model.[Results]We examined the proposed model with 1559 news articles from the website of Southern Weekly.When the number of extracted keywords was three,the model’s keyword extraction precision values were 4.7%and 6.5%higher than those of the original Text Rank and TF-IDF algorithms.[Limitations]The fusion algorithm increased computational complexity.[Conclusions]The proposed algorithm could extract keywords more effectively.