基于N元语法的英文学术文献聚类标签抽取算法

N-gram Based on Cluster Label Extracting Algorithm for English Paper

提出一种基于N元语法的英文学术文献聚类标签抽取算法,该算法利用N元语法在大规模语料库上进行先期学习生成领域短语词表,再通过K-means算法进行聚类,从聚簇中抽取N元语法项计算TFIDF值,对出现在词表中的特征项赋以更高的权值,以得分最高的特征项作为聚类标签。实验结果表明,该算法能获得更好的实验效果。同时,在抽取聚类标签时提出一种改进的TFIDF权重计算,在评价标签质量时提出一种新的标签评价方法R@N方法。

In this paper, a novel cluster label extracting algorithm for English paper based on N - gram is proposed. Before the clustering, this algorithm first uses N - gram to generate the field phrases list by prior learning in the large - scale corpus, then clusters the English paper using K - means algorithm. Finally, the highest score N - gram terms from the cluster is extracted as the label. In the score calculation, if the term exists in the field phrases list, it is set double weight. Experimental resuhs show that the quality of cluster label is improved. Furthermore, an improved TFIDF calculation method is developed, and a new R@ N method to evaluate the cluster label is proposed.