基于句法分析及主题分布的关键词抽取模型

Keyword extraction model based on syntactic analysis and topic distribution

针对TextRank算法在抽取篇章关键词时忽略句法信息、主题信息等问题,提出基于句法分析与主题分布的篇章关键词抽取模型(S-TAKE)。模型分为段落和篇章两阶段递进抽取篇章关键词,首先以段落为单位,结合词共现、语法及语义信息抽取段落关键词;然后根据段落主题对段落聚类,形成段落主题集;最后根据段落主题分布特征抽取篇章关键词。在公开的新闻数据集上,模型的抽取效果较原始TextRank提升了约10%。实验结果表明,S-TAKE的抽取效果有了明显提升,证明了语法信息及主题信息的重要性。

Aiming at the problem that TextRank ignored syntactic information and topic information when extracting chapter keywords,this paper proposed a chapter keyword extraction model based on syntactic analysis and topic distribution(S-TAKE).This model included two stages of chapter keyword extraction,such as paragraph and chapter.Firstly,it used paragraphs as a unit to extract paragraph keywords by combining word co-occurrence,grammatical and semantic information.Then it clustered the paragraphs according to the paragraph topics to form the paragraph topic set.Finally,it extracted chapter keywords based on the distribution characteristics of paragraph topics.On the open news dataset,the model’s extraction effect improved by about 10%compared with the original TextRank.Results show that S-TAKE model has significantly improved the extraction effect,and proves the importance of grammatical information and topic information.