Review文献类型标注对科研评价结果的影响

The Influence of Review’s Document Type Marking on the Results of Research Evaluation

[目的/意义]为了提升论文文献类型标签的准确性,了解论文文献类型标签准确性对科研评价结果的影响大小,[方法/过程]该文选取9个系列的159本Review期刊在2017-2018年发表的论文作为研究对象,对比分析了这些论文在Web of Science、Scopus和期刊官网上文献类型标注的差异,并通过召回率、准确率和F1-score三个指标对差异进行测度,并选取期刊Annual Review of Analytical Chemistry作为典型案例,分析其文献类型标签改变对文献类型归一化指标CNCI的影响程度。[结果/结论]研究发现不同出版商的Review系列期刊和不同类型Review期刊在不同数据库中的文献类型标注准确率存在差异,案例分析发现CNCI对文献类型标签变化极为敏感,基于CNCI的指标,如JCI也较为敏感。因此,为了保证科研评价指标的准确性,文献类型标签的准确性很重要,而为了更好实现文献类型归一化,发展和使用更为鲁棒性的指标是关键。

[Objective/Significance]In order to improve the accuracy of paper type marking and understand the impact of the accuracy of paper type marking on the results of research evaluation,[Method/Process]this paper selects 159 papers published in nine series of review journals from 2017 to 2018 as the research object,compares and analyzes the differences of these papers in document type marking on Web of Science,Scopus and the official website of journals,and measures the differences through three indicators:recall rate,accuracy rate and F1 score,and the journal Annual Review of Analytical Chemistry is selected as a typical case to analyze the influence of the change of the document type marking on CNCI,a document type normalization index.[Results/Conclusions]The study found that there were differences in the accuracy of document type marking between review series journals of different publishers and review journals of different types in different databases.The case study found that CNCI was extremely sensitive to changes in document type mark,and CNCI based indicators,such as JCI,were also more sensitive.Therefore,in order to ensure the accuracy of research evaluation indicators,the accuracy of document type marking is very important,and in order to better realize the normalization of document types,developing and using more robust indicators is the key.