国内在线作文自动评分系统的效度研究——以句酷作文网为例

A Study on the Reliability of Online Autonomous Scoring System in China——Exemplified by Juku Correcting Network

通过为期近一年的实证研究,对"句酷网"给受试在四种体裁作文任务中的自动评分和人工评分结果进行了对比分析。这是在以往相关实证研究的基础上,探索性地通过较大样本数据,验证"句酷网"对不同体裁作文的评分效度。实验结果表明:"句酷网"的批改成绩和教师的人工批改成绩显著相关,但前者对不同体裁的作文评分效度存在显著差异,其中对观点论证型的作文的评分结果和人工评分结果的相关性最高。本实验呈现的相关性虽然存在差异,但基本体现出"句酷网"的评分结果的前后一致性和稳定性。同时,在使用"句酷网"一学年后,学生的写作成绩也显著提高。

Through an academic-year experimental research, a comparison has been made be- tween the Juku Correcting Network＇s scoring and teachers＇ scoring of subjects＇ writings in four tasks of different styles. Based on the limited previous empirical studies, this study examines the reliability of Juku Correcting Network through a relatively larger sample. Findings indicated that the results of Juku Correcting Network and the teachers＇ correcting are significantly positively correlated, and the positive correlation is most significant in the writing task of argument style in the current study. Although the online autonomous scoring results in the four writing tasks do not all show significant positive correlation in the within-groups, the fact revealed that there is reliable consistency and stability in the autonomous scoring results of Juku Correcting Network. Moreover, the use of Juku Correcting Network can signifi- cantly enhance subjects＇ English writing abilities after an academic year.