基于多面Rasch模型的作文网上评卷“趋中评分”判定研究

Research on Identifying Central Rating in Net-based Composition Scoring Using Many-facet Rasch Model

写作测试则是评价学生语言能力最有效的工具之一,但在作文评分中,评分员容易出现"趋中评分",即打"中间分"的现象。趋中评分是评分过程中形成的一种系统性误差,它在一定程度上会影响作文评分的质量。本文基于多面Rasch模型,采用MHK(三级)考试的实测数据,估计了考生的能力值和期望得分,计算出残差标准差、残差与期望得分的相关,同时参考原始评分的峰度值、评分员infit值、评分员期望评分的峰度值和专家校验卷的分析结果,最后确定了具有趋中评分倾向的评分员,同时提出了判定趋中评分的方法和标准。

Researchers have found that in composition scoring process, some raters tended to assign central scores, avoiding using the high or low end of the rating scale. This is the so-called ＂central rating＂. This study tried to identify central raters in net-based composition scoring using Many-Facet Rasch Model. The statistics were collected from MHK （Level III） composition rating which contained 15,194 examinees and 45 raters. This research estimated each examinee＇ s ability and expected score, and then by computing the residual＇ s standard deviation and coefficient with expected score, kurtosis of observed scores, infit statistics generated by the model, kurtosis of expected ratings and analyzing the result of expert-consensus scores, researchers successfully identified two ＂suspected central rater＂. Furthermore, this study suggested a series of standards to identify central rating which could be applied in future net-based composition scoring.