基于TextRank的关键词提取改进方法研究

Research on Improved Method of Keyword Extraction Based on TextRank

基于TextRank的关键词提取方法主要考虑文档的语义信息,没有考虑到文本的结构信息,导致关键词提取的准确率较低。为了得到更好的关键词提取效果,论文对新闻语料库关键词提取方法进行了研究,提出将文本的结构信息和语义信息相结合。基于TextRank算法,利用文本的结构信息(词语位置、词语跨度)和词语相似度改变词语权重计算方式,进而提取文本关键词。将论文提出的算法与TF-IDF、TextRank、Word2vec+TextRank进行了对比实验,结果表明,从准确率P、召回率R、测量值F三个方面都取得了良好的实验效果。具体表现为当关键词个数相同K=3时,TextRank算法评价指标准确率P、召回率R、测量值F分别为22.21%、19.78%、20.92%;加入文档的词语位置信息和词语跨度信息以后评价指标准确率P、召回率R、测量值F分别为29.83%、31.34%、30.56%。

The keyword extraction method based on TextRank mainly considers the semantic information of the document,and does not consider the structural information of the text,resulting in a low accuracy of keyword extraction.The present study aims to improve the keywords extraction effect of news articles by studying the Keywords extraction methods based on TextRank.The paper extracts the keywords by suing the structural information of documents and words similarities which change the calculation methods of keyword weight.The experiment of this study achieves the better results from three aspects,which are accuracy rate P,recall rate R and measured value F,by comparing the method with TF-IDF,TextRank,combined Word2vec and TextRank.The specific performance is as follows when the number of keywords is the same as K=3,the evaluation indexes P,R and F of TextRank algorithm are 22.21%,19.78%and 20.92%respectively.After adding the document structure information,the evaluation indexes P,R and F are 29.83%,31.34%,30.56%.