智能评分技术应用效果的评价研究

Research on the Effectiveness Evaluation of Intelligent Scoring Applications

智能评分技术在教育考试实践中应用越来越广泛,但社会对于其准确性和安全性问题还存有疑虑,其中一个重要原因是当前尚缺少智能评分应用效果的评价方法。针对于此,提出智能评分的研究和应用框架,在文献分析和实践经验基础上提出面向机器学习专家和面向领域评分专家的评价指标,形成一套智能评分应用效果综合评价方案。以军队文职人员招录考试智能评分数据为案例,利用综合评价方案中的统计指标对其评分效果进行了验证。研究结果显示,机器评分的一致性和稳定性均高于人工评分,能够起到良好的辅助质检作用。

Intelligent scoring technology is increasingly being applied in educational examinations,but there are still many doubts about its accuracy and security in the society.One of important reasons is that there is currently a lack of evaluation method for the effectiveness of the intelligent scoring.To address this problem,we propose a research and application framework for intelligent scoring and evaluation metrics for machine learning experts and domain-specific scoring experts based on literature analysis and practical experience,and form a comprehensive evaluation scheme for the effectiveness of intelligent scoring.Taking the military civilian recruitment examination as a case,this study evaluated and verified the effectiveness of intelligent scoring using the proposed indicators.The results show that the consistency and stability of intelligent scoring are higher than manual scoring.Intelligent scoring contributes to improving the effectiveness of quality inspection.