智能作文评价的效果研究

Study on the Effect of Intelligent Essay Evaluation

本研究以智能作文评价系统为研究对象,选取149篇中学生作文为样本,从评分有效性、反馈类型与层次和反馈精准度等三方面进行了智能评价作文的比较研究,结果发现:1)智能作文评价与教师作文评分有较高的一致性和显著相关性,评分效度良好;2)相较于教师评价作文,智能作文评价的表扬型反馈占比较高,直接反馈和提问反馈占比较低;两种作文评价中的高阶写作技能反馈占比都高于低阶写作技能反馈;3)两种作文评价的反馈精确度基本持平,智能作文评价的反馈召回率要高于教师作文评价,二者的召回率都有较大的提升空间。比较可知,智能作文评价作为传统教师作文评价的有益补充,为个性化的作文教学指导提供了可能,两种作文评价的反馈信息各有侧重,教师和智能作文评价系统的优势互补、人机协同有望突破传统写作教学的困境。

This study took IN Class Intelligent Essay Evaluation as a research object,selected 149 essays from junior and high school as research samples to verify the effectiveness of the Intelligent Evaluation System.The human and machine methods were compared from three aspects:scoring validity,feedback type with level,and feedback accuracy.The study found that:1)Two scorings have a high consistency and significant correlation,and the validity of the score is good;2)Compared with teachers’essay evaluation,the proportion of praise feedback in Intelligent Essay Evaluation is higher,while the proportion of direct feedback and question feedback is lower;the proportion of high-order feedback in both forms is higher than that of low-order feedback;3)The precision of the two kinds of essay evaluation is almost the same,but the recall rate of Intelligent Essay Evaluation is higher than that of teacher essay evaluation,there is a large room for improvement in recall rates for both.The results show that Intelligent Essay Evaluation can be a positive supplement to teachers’essay evaluation,and it provides a possibility for personalized writing instruction.The feedback types of the two evaluation methods have their own focuses.With its complementary advantages,man-machine cooperation between teachers and the Intelligent Essay Evaluation system is expected to break through the dilemma of traditional writing teaching.