全过程多主体负责任评价的概念、内涵和关键问题

The Concept, Connotation and Key Issues of Responsible Evaluation based on Multi-agent and Whole-process

负责任评价在国际科技界逐渐兴起并得到关注,但是关于负责任评价的概念、内涵等理论研究尚属空白。通过总结国际上代表性学术组织(DORA, HEFCE, INORMS)在推动负责任评价上的相关探索,进一步基于"目标—要素—主体—准则"提出全程多主体负责任评价(REMAP)的概念、揭示了负责任评价的内涵、提出了负责任评价的4项基本准则(ATTA)和8个关键问题,以期为新时期破"四唯"下的我国科研管理与评价实践工作提供启示与借鉴。

In early 2020, the release of relevant documents from the Ministry of Science and Technology and the Ministry of Education further strengthened the guidance of breaking the "Siwei ", thus the voice of the scientific and technological community looking forward to building new theories and methods of scientific research evaluation is growing. From the perspective of practice at home and abroad, responsible evaluation is quietly rising in the global academic circles and has received more attention. Many well-known international academic organizations have explored the concept and connotation of responsible evaluation at different levels. However, how to understand the concept and connotation of responsible evaluation has not yet formed a systematic understanding in theory, let alone effective guidance for practical work. In order to enrich the research in this field, based on the idea of "responsible innovation", this paper summarizes the relevant exploration of international representative academic organizations(DORA, HEFCE, INORMS) in promoting responsible evaluation, puts forward the concept of whole process multi-agent responsible evaluation(REMAP) based on "target-element-subject-criterion", and reveals the connotation of responsible evaluation as well as four basic criteria(ATTA) and eight key issues of responsible evaluation. From the target level, REMAP is the extension and expansion of responsible innovation in the field of science and technology evaluation. It advocates the scientific research evaluation oriented by innovation quality, performance and contribution, and opposes the quantitative evaluation of simple machinery;It emphasizes the "responsibility" consciousness of key subjects in scientific and technological innovation activities-in the whole process of promoting and guiding scientific and technological innovation activities, we should consider not only the short-term impact, but also the positive and negative impact on all aspects of economic and social development from the perspective of future development;From the element level, REMAP is not only limited to the organization of evaluation,but also runs through the whole process of scientific research evaluation, including evaluation purpose, method design, expert selection, result use, etc;From the perspective of subject level, responsible evaluation is not only limited to scientific researchers, but also involves the key stakeholders who organize or participate in scientific and technological evaluation.Scientific research institutions, innovation groups, business circles, policy makers and research funders, third-party evaluation institutions, evaluation experts and other groups should establish a sense of responsibility and responsibility, and consciously assume their due responsibilities in scientific and technological innovation activities;From the perspective of criteria, REMAP should meet the four criteria of "Assessable, Testable, Traceable and Accountable"(ATTA). It not only ensures and promotes the implementation of responsible evaluation, but also supervises and feeds back the implementation of responsible evaluation in practice. Therefore, it can be subdivided into eight key issues in the application of responsible evaluation in practice. Specifically, from the perspective of evaluable principle, one should consider what is the purpose of the proposed scientific research evaluation? Is scientific research evaluation the best way to solve problems? From the perspective of testability principle, one should consider how to make good use of quantitative tools and quantitative evaluation methods? How to select experts and make good use of peer review? From the perspective of traceability principle,one should consider whether the responsibility division of each evaluation subject is clear and the boundary is clear? Is the evaluation result based on a complete chain of evidence? From the perspective of the principle of accountability, one should consider who is responsible for the evaluation results and the use of the results? What can be further improved in the whole assessment? Furthermore, combined with the theoretical research on remap, this paper puts forward some policy suggestions on how to implement responsible evaluation in China.