Guidelines on clinical research evaluation of artificial intelligence in ophthalmology(2023)

With the upsurge of artificial intelligence(AI)technology in the medical field,its application in ophthalmology has become a cutting-edge research field.Notably,machine learning techniques have shown remarkable achievements in diagnosing,intervening,and predicting ophthalmic diseases.To meet the requirements of clinical research and fit the actual progress of clinical diagnosis and treatment of ophthalmic AI,the Ophthalmic Imaging and Intelligent Medicine Branch and the Intelligent Medicine Committee of Chinese Medicine Education Association organized experts to integrate recent evaluation reports of clinical AI research at home and abroad and formed a guideline on clinical research evaluation of AI in ophthalmology after several rounds of discussion and modification.The main content includes the background and method of developing this guideline,an introduction to international guidelines on the clinical research evaluation of AI,and the evaluation methods of clinical ophthalmic AI models.This guideline introduces general evaluation methods of clinical ophthalmic AI research,evaluation methods of clinical ophthalmic AI models,and commonly-used indices and formulae for clinical ophthalmic AI model evaluation in detail,and amply elaborates the evaluation methods of clinical ophthalmic AI trials.This guideline aims to provide guidance and norms for clinical researchers of ophthalmic AI,promote the development of regularization and standardization,and further improve the overall level of clinical ophthalmic AI research evaluations.

Unequal Distribution of Innovation Efforts for Neglected Tropical Diseases: The Role of Funding Evaluation Criteria

Background: International research and innovation efforts for neglected tropical diseases have increased in recent decades due to disparities in overall health research funding in relation to global burden of disease. However, within the field of neglected tropical diseases some seem far more neglected than others. In this research the aim is to investigate the distribution of resources and efforts, as well as the mechanisms that underpin funding allocation for neglected tropical diseases. Methodology: A systematic literature review was conducted to establish a comprehensive overview of known indicators for innovation efforts related to a wide range of neglected tropical diseases. Articles were selected based on a subjective evaluation of their relevance, the presence of original data, and the breadth of their scope. This was followed by thirteen in-depth open-ended interviews with representatives of private, public and philanthropic funding organizations, concerning evaluation criteria for funding research proposals. Results: The findings reveal a large difference in the extent to which the individual diseases are neglected with notable differences between absolute and relative efforts. Criteria used in the evaluation of research proposals relate to potential impact, the probability of success and strategic fit. Private organizations prioritize strategic fit and economic impact;philanthropic organizations prioritize short-term societal impact;and public generally prioritize the probability of success by accounting for follow-up funding and involvement of industry. Funding decisions of different types of organizations are highly interrelated. Conclusions: This study shows that the evaluation of funding proposals introduces and retains unequal funding distribution, reinforcing the relative neglect of diseases. Societal impact is the primary rationale for funding but application of it as a funding criterion is associated with significant challenges. Furthermore, current application of evaluation criteria leads to a primary focus on short-term impact. Through current practice, the relatively most neglected diseases will remain so, and a long-term strategy is needed to resolve this.

Evaluation of the effectiveness of preventive nursing measures for pressure injuries in patients in the neurology intensive care unit

BACKGROUND Patients in neurology intensive care units(ICU)are prone to pressure injuries(PU)due to factors such as severe illness,long-term bed rest,and physiological dysfunction.PU not only causes pain and complications to patients,but also increases medical burden,prolongs hospitalization time,and affects the recovery process.AIM To evaluate and optimize the effectiveness of pressure injury prevention nursing measures in neurology ICU patients.METHODS A retrospective study was conducted,and 60 patients who were admitted to the ICU of the Department of Neurology were selected and divided into an observation group and a control group according to the order of admission,with 30 people in each group.The observation group implemented pressure injury prevention and nursing measures,while the control group adopted routine care.RESULTS Comparison between observation and control groups following pressure injury prevention nursing intervention revealed significantly lower incidence rates in the observation group compared to the control group at 48 h(8.3%vs 26.7%),7 d(16.7%vs 43.3%),and 14 d(20.0%vs 50.0%).This suggests a substantial reduction in pressure injury incidence in the observation group,with the gap widening over time.Additionally,patients in the observation group exhibited quicker recovery,with a shorter average time to get out of bed(48 h vs 72 h)and a shorter average length of stay(12 d vs 15 d)compared to the control group.Furthermore,post-intervention,patients in the observation group reported significantly improved quality of life scores,including higher scores in body satisfaction,feeling and function,and comfort(both psychological and physiological),indicating enhanced overall well-being and comfort following the implementation of pressure injury prevention nursing measures.CONCLUSION Implementing pressure injury preventive care measures for neurology ICU patients will have better results.

Comprehensive applicability evaluation of four precipitation products at multiple spatiotemporal scales in Northwest China

Precipitation plays a crucial role in the water cycle of Northwest China.Obtaining accurate precipitation data is crucial for regional water resource management,hydrological forecasting,flood control and drought relief.Currently,the applicability of multi-source precipitation products for long time series in Northwest China has not been thoroughly evaluated.In this study,precipitation data from 183 meteorological stations in Northwest China from 1979 to 2020 were selected to assess the regional applicability of four precipitation products(the fifth generation of European Centre for Medium-Range Weather Forecasts(ECMWF)atmospheric reanalysis of the global climate(ERA5),Global Precipitation Climatology Centre(GPCC),Climatic Research Unit gridded Time Series Version 4.07(CRU TS v4.07,hereafter CRU),and Tropical Rainfall Measuring Mission(TRMM))based on the following statistical indicators:correlation coefficient,root mean square error(RMSE),relative bias(RB),mean absolute error(MAE),probability of detection(POD),false alarm ratio(FAR),and equitable threat score(ETS).The results showed that precipitation in Northwest China was generally high in the east and low in the west,and exhibited an increasing trend from 1979 to 2020.Compared with the station observations,ERA5 showed a larger spatial distribution difference than the other products.The overall overestimation of multi-year average precipitation was approximately 200.00 mm and the degree of overestimation increased with increasing precipitation intensity.The multi-year average precipitation of GPCC and CRU was relatively close to that of station observations.The trend of annual precipitation of TRMM was overestimated in high-altitude regions and the eastern part of Lanzhou with more precipitation.At the monthly scale,GPCC performed well but underestimated precipitation in the Tarim Basin(RB=-4.11%),while ERA5 and TRMM exhibited poor accuracy in high-altitude regions.ERA5 had a large bias(RB≥120.00%)in winter months and a strong dispersion(RMSE≥35.00 mm)in summer months.TRMM showed a relatively low correlation with station observations in winter months(correlation coefficients≤0.70).The capture performance analysis showed that ERA5,GPCC,and TRMM had lower POD and ETS values and higher FAR values in Northwest China as the precipitation intensity increased.ERA5 showed a high capture performance for small precipitation events and a slower decreasing trend of POD as the precipitation intensity increased.GPCC had the lowest FAR values.TRMM was statistically ineffective for predicting the occurrence of daily precipitation events.The findings provide a reference for data users to select appropriate datasets in Northwest China and for data developers to develop new precipitation products in the future.

Automated writing evaluation(1995-2022):Research trends and implications

Automated writing evaluation is the use of computer programs to evaluate essays written in an educational setting.This study aims to evaluate the state of the art and the trends of automated writing evaluation research.A total of 206 automated writing evaluation-related publications published in 88 journals between 1995 and 2022 were used as the data in our study.Bibliometric methods were employed to identify the publication trends,venues for publication,productive authors,highly cited articles,seminal works,and the most frequently explored topics across the past decades.The findings show that automated writing evaluation has become a fast-growing interdisciplinary area of interest.In addition,this line of research has witnessed a growing interest in topics pertaining to the latest technology,the perception of children,the teachers’feedback,and complex tasks.Implications of our study for future research are also discussed.

Impact of Non-Clinical Years on Surgery Residents’ Technical Skills: Evaluation of a Technical Skills Refresher Curriculum

We ascertained the opinions of residents and faculty regarding technical skills decay during non-clinical training years and evaluated the effectiveness of a technical skills refresher curriculum (TSRC) offered to residents in the month prior to rejoining clinical training years. 32 faculty and 14 residents completed surveys which gleaned opinions regarding technical skills decay during non-clinical years. Six residents completed a TSRC during the month prior to rejoining clinical training. We compared clinical evaluations of the residents who completed the TSRC to residents who rejoined clinical training prior to the implementation of the curriculum. Surveys indicated that residents and faculty believe that non-clinical years have a slightly negative impact on technical skills and residents who completed dedicated research years would require up to 4 months for technical skills to return to the level of their non-research peers. Residents who completed the TSRC reported having significantly higher comfort levels with their technical skills after the curriculum

Technology Research on Lightning Strike Risk Evaluation of a Cable Car

According to structure function and lightning damage of a cable car, a feasible method of lightning strike risk evaluation for a cable car was put forward based on the evaluation model and evaluation method in the standard IEC62305-2. According to the difference between common buildings and cable cars, problems of height non-uniformity of equivalent section caused by inclination of the cable car and diversity of lightning activity regularity caused by the large area were resolved, and expected annual average frequency of lightning strike was calculated using three dimensional graphic approach and regional lightning characteristic analysis. Based on different types of damage process and loss consequences, according to interception effect against lightning invasion of the lightning protection measures and the method of probability selection proposed in the standard, the probability of casu- alty caused by direct lightning strike in a cable car and a waiting area as well as probabilities of casualty caused by failure of electronic information systems were cal- culated.

A Research Commentary and Evaluation

Language teaching is both a science and an art. As a science, it has its own principles and laws to ensure the operation;as an art,it combines the inspiration,creativity and other affection factors together. To many scholars and language teachers, during their studies on language teaching and linguistics, they adopt many research methods to complement the studies and find some useful ways to improve the language teaching. In this paper,it is aimed to analyze the different methodologies adopted by the two different authors in two papers and make a commentary and evaluation.

Research Progress on Evaluation Frameworks of Regional Ecological Sustainability

As natural ecosystems provide the material basis and fundamental support for regional sustainable devel-opment,the sustainability of natural ecosystems is an important prerequisite and a viable approach for the achievement of regional sustainable development.It is also the final criteria to assess whether sustainable development paradigm is successful.Along with the increasing impacts of human activities on natural ecosystems,the evaluation of regional ecological sustainability has become one of the key issues for research on macro ecology and sustainable development.Based on different unit of indicators,this study firstly groups the evaluation frameworks of regional ecological sus-tainability into three major types:comprehensive index evaluation with dimensionless unit,monetary valuation,and biophysical quantity measurement.We then discuss and compare these types in terms of basic principles,scope of ap-plications,advantages and shortcomings.Finally,drawn on the discussion about characteristics of ecological sustain-ability,we outline the current trend and future directions of regional ecological sustainability evaluation,for instance,transition from sustainable development evaluation to sustainability science,integration of goal-oriented and problem-solving approaches,combination of spatial pattern analysis and ecological sustainability evaluation,and en-hancement of ecological sustainability evaluation at landscape scale.

Patent Citations Analysis and Its Value in Research Evaluation： A Review and a New Approach to Map Technology-relevant Research

Purpose： First, to review the state-of-the-art in patent citation analysis, particularly characteristics of patent citations to scientific literature （scientific non-patent references, SNPRs）. Second, to present a novel mapping approach to identify technology-relevant research based on the papers cited by and referring to the SNPRs. Design/methodology/approach： In the review part we discuss the context of SNPRs such as the time lags between scientific achievements and inventions. Also patent-to-patent citation is addressed particularly because this type of patent citation analysis is a major element in the assessment of the economic value of patents. We also review the research on the role of universities and researchers in technological development, with important issues such as universities as sources of technological knowledge and inventor-author relations. We conclude the review part of this paper with an overview of recent research on mapping and network analysis of the science and technology interface and of technological progress in interaction with science. In the second part we apply new techniques for the direct visualization of the cited and citing relations of SNPRs, the mapping of the landscape around SNPRs by bibliographic coupling and co-citation analysis, and the mapping of the conceptual environment of SNPRs by keyword co-occurrence analysis. Findings： We discuss several properties of SNPRs. Only a small minority of publications covered by the Web of Science or Scopus are cited by patents, about 3%-4%. However, for publications based on university-industry collaboration the number of SNPRs is considerably higher, around 15%. The proposed mapping methodology based on a ＂second order SNPR approach＂ enables a better assessment of the technological relevance of research. Research limitations： The main limitation is that a more advanced merging of patent and publication data, in particular unification of author and inventor names, in still a necessity. Practical implications： The proposed mapping methodology enables the creation of a database of technology-relevant papers （TRPs）. In a bibliometric assessment the publications of research groups, research programs or institutes can be matched with the TRPs and thus the extent to which the work of groups, programs or institutes are relevant for technological development can be measured. Originality/value： The review part examines a wide range of findings in the research of patent citation analysis. The mapping approach to identify a broad range of technologyrelevant papers is novel and offers new opportunities in research evaluation practices.

Identification and Evaluation of Low Resistivity Pay Zones by Well Logs and the Petrophysical Research in China

This paper presents an overview of petrophysical research and exploration achievements of low resistivity pay （LRP） zone by well logs in China. It includes geological characteristics and characteristics of well log response of the low resistivity pay zones discovered and evaluated in recent years, as well as the problems in recognizing and evaluating low resistivity pay zones by well logs. The research areas mainly include the Neogene formations in the Bohai Bay Basin, the Triassic formations in the northern Tarim Basin and the Cretaceous formations in the Junggar Basin, The petrophysical research concerning recognition and evaluation of the low resistivity pays, based on their genetic types, is introduced in this paper.

Redesigning the Model of Book Evaluation in the Polish Performance-based Research Funding System

Purpose: This study aims to present the key systemic changes in the Polish book evaluation model to focus on the publisher list, as inspired by the Norwegian Model. Design/methodology/approach: In this study we reconstruct the framework of the 2010 and 2018 models of book evaluation in Poland within the performance-based research funding system. Findings: For almost 20 years the book evaluation system in Poland has been based on the verification of various technical criteria(e.g. length of the book). The new 2018 model is based on the principle of prestige inheritance(a book is worth as much as its publisher is) and is inspired by the publisher list used in the Norwegian Model. In this paper, we argue that this solution may be a more balanced policy instrument than the previous 2010 model in which neither the quality of the publisher nor the quality of the book played any role in the evaluation.Research limitations: We work from the framework of the 2018 model of book evaluation specified in the law on higher education and science from 20 July 2018, as implementation acts are not available yet. Practical implications: This study may provide a valuable point of reference on how structural reforms in the research evaluation model were implemented on a country level. The results of this study may be interesting to policy makers, stakeholders and researchers focused on science policy. Originality/value: This is the very first study that presents the new framework of the Polish research evaluation model and policy instruments for scholarly book evaluation. We describe what motivated policy makers to change the book evaluation model, and what arguments were explicitly raised to argue for the new solution.

Google Scholar University Ranking Algorithm to Evaluate the Quality of Institutional Research

Education quality has undoubtedly become an important local and international benchmark for education,and an institute’s ranking is assessed based on the quality of education,research projects,theses,and dissertations,which has always been controversial.Hence,this research paper is influenced by the institutes ranking all over the world.The data of institutes are obtained through Google Scholar(GS),as input to investigate the United Kingdom’s Research Excellence Framework(UK-REF)process.For this purpose,the current research used a Bespoke Program to evaluate the institutes’ranking based on their source.The bespoke program requires changes to improve the results by addressing these methodological issues:Firstly,Redundant profiles,which increased their citation and rank to produce false results.Secondly,the exclusion of theses and dissertation documents to retrieve the actual publications to count for citations.Thirdly,the elimination of falsely owned articles from scholars’profiles.To accomplish this task,the experimental design referred to collecting data from 120 UK-REF institutes and GS for the present year to enhance its correlation analysis in this new evaluation.The data extracted from GS is processed into structured data,and afterward,it is utilized to generate statistical computations of citations’analysis that contribute to the ranking based on their citations.The research promoted the predictive approach of correlational research.Furthermore,experimental evaluation reported encouraging results in comparison to the previous modi-fication made by the proposed taxonomy.This paper discussed the limitations of the current evaluation and suggested the potential paths to improve the research impact algorithm.

Practice and Challenge of International Peer Review: A Case Study of Research Evaluation of CAS Centers for Excellence

Purpose: The main goal of this paper is to show that international peer review can work in China’s context with satisfactory outcomes. Moreover, this paper also provides a reference for the practice of science and technology management.Design/methodology/approach: This paper starts with a discussion of two critical questions about the significance and design of international peer review. A case study of international peer review of CAS Centers for Excellence is further analyzed. Findings: International peer review may provide a solution to address the problem of quantitative oriented research evaluation in China. The case study of research evaluation of CAS Centers for Excellence shows that it is possible and feasible to conduct an international peer review in China’s context. When applying this approach to other scenarios, there are still many issues to consider including individualized design of international peer review combined with practical demands, and further improvement of theories and methods of international peer review. Research limitation: 1) Only the case of international peer review of CAS Centers for Excellence is analyzed;2) A relatively small number of respondents were surveyed in the questionnaire. Practical implications: The work presented in this study can be used as a reference for future studies.Originality/value: Currently, there are no similarly detailed studies exploring the significance and methodology of international peer review in China.

Research and Evaluation on Key Technological Indicators for Airborne and Shipborne Gravimetry

Gravimetry technical guides are the scientific basis for air-sea gravimetry. However, the existing technical guides in China are behind the application requirements. This study analyzed the most important indicators of air-sea gravimetry, including the density of survey lines, gravimetry accuracy and space resolution, stability and reliability of the air-sea gravimeter, and proposed a gravimetry accuracy assessment system consisting of gravity RMS of error, systematic error and mean error, and an assessment system for the gravimeter stability consisting of the relative accuracy of the scale value, monthly zero-drift, RMS of the monthly nonlinear zero-drift variation and the threshold of the monthly nonlinear zero-drift variation. The mathematic models for the measurement point determination in shipborne gravimetry, E tv s correction for airborne gravimetry, platform tilt correction and evaluation air-sea gravimetry were also analyzed and modified. This work will provide technology support for the composition of the military-civil air-sea gravimetry technical guides.

The theory and method research on the double evaluation of land and space:A survey

By reviewing the theories and methods of resource and environmental carrying capacity evaluation and suitability evaluation of land and space development,it provides theoretical and technical support for rational layout and optimization of land and space development pattern.From the aspects of resource carrying capacity,environmental carrying capacity,the importance of ecological protection,the suitability of agricultural production,and the suitability of urban construction,we sort out domestic and foreign documents.The evaluation of resource carrying capacity has changed from single-factor evaluation to multi-factor comprehensive evaluation.At present,the optimization of land use pattern based on the“three-generation”space is still under exploration,it is not enough to systematize and summarize the thought of optimizing the spatial pattern of the national territory.The theoretical framework,model,method and index of“dual evaluation”of territorial space need to be improved continuously,it also needs the scientific analysis and cognition from the natural direction to the human location direction and the spatial equilibrium state as the new driving force.

Evaluations on social network analysis (SNA) research in China's Mainland:A critical assessment of journal literatures

Social network analysis(SNA) has been introduced to China's Mainland since the end of last century. It is often stated that SNA research has experienced rapid growth in China over these years, but few studies have been conducted to prove the statement. This paper aims at exploring the research status and development of SNA in China by a critical assessment of journal articles. Our findings show that SNA is an evolving and diversified research area which has rich themes and topics, and can be applied to those studies on different levels, context and disciplines, and attract researchers and scholars from various fields and domains. In addition, the information community(Library & Information Science and Information Systems) plays a leading role in the SNA related researches. The paper also points out the research on SNA in China has some limitations, so it proposes several implications for the future development of SNA research from perspectives of information science.

An Action Research on Constructing a Multiple Learning Evaluation System on the Context of College English Teaching Reform

Learning evaluation reform, an important part of the College English Teaching Reform, is not only a basis for teachers to reflect on their teaching method, to improve their teaching management, and to ensure the teaching quality, but also an effective means for students to enhance their learning efficiency and to gain sound learning effect as well. This research was intended to apply action research to the construction of multiple learning evaluation system with the aim of proving the effectiveness of the system on raising students＇ learning autonomy to find problems in the teaching, to testify the problems through questionnaire, then to design a feasible teaching strategy and carry out an experimental teaching, and finally to reflect on the whole teaching process from the perspective of how well the learner＇s learning autonomy was enhanced. The research result is valid, positively testifying the presupposition that the Multiple Evaluation System is a comprehensive one, which makes it possible to evaluate the students＇ progress in the whole learning process.

Research and perspectives on criteria for evaluation of nursing research achievements

This paper introduces concepts related to scientific research achievements, analyzes current evaluation methods with regard to nursing research achievements and their application at home and abroad, and summarizes findings from the investigation of obstacles to the application of nursing research results in China, aiming to provide reference points for the evaluation and application of nursing research results in China.

Research on Scientific Research Performance Evaluation in Colleges and Universities Based on Malmquist Index

Scientific research is of vital importance to the development of colleges and universities.Scientific evaluation of the scientific research performance of colleges and universities,especially the study of their dynamic performance in different time ranges,can not only promote the progress of scientific research in colleges and universities,but also contribute to the rational allocation of national educational resources.Based on the Malmquist index,using the panel data of the five-year period from 2012 to 2016,this paper evaluates the changes in the research performance of China’s colleges and universities,analyzes the influencing factors of total factor productivity changes from three perspectives of time series,comparison of colleges and universities,and regional analysis,and proposes countermeasures and suggestions.