基于随机森林和LDA的论文自动分类及主题挖掘研究

Research on automatic classification and topic mining of thesis based on LDA and random forest

当前科研成果呈爆炸式增长,论文跨学科交叉分布不断深化,精准获取所需的论文需要耗费大量的时间和精力。文章提出一种基于随机森林的论文自动分类方法,实现对海量论文的自动分类;提出一种基于LDA模型的主题挖掘方法,提取论文关键词并进行词云展示。实验数据采用Selenium技术抓取中国知网九大主题的1710篇论文,实验结果表明,该论文分类方法在准确率、召回率和F值上都有所提升,有效地挖掘出各学科的主题词,为下一步引文分析、文本挖掘和知识图谱构建提供有效支撑。

With the explosive growth of scientific research results, the cross-disciplinary distribution of papers has been deepened,and it takes a lot of time and effort to accurately acquire the required papers. In this paper, an automatic classification methodbased on random forest is proposed to realize the automatic classification of massive papers and a topic mining method based onLDA model is proposed to extract the keywords of the paper and display the word cloud. The experimental data used Seleniumtechnology to capture 1710 papers on the nine themes of the CNKI. The experimental results show that the paper classificationmethod has improved the precision, recall and F-measure, effectively mining the subjects of various disciplines. The keywordsextracted provide effective support for the next citation analysis, text mining and knowledge graph construction.