分步增值评分——提高主观题评分质量的有效方法

Multistage Rating Augmentation——An Effective Way to Improve Subjective Performance Rating

主观题特别是作文题评分的客观性和有效性存在着一定的问题,有必要通过优化评分流程降低作文评分的误差程度。本研究通过多面Rasch模型对500份国家级考试作文题的评分进行了分析,比较了分步增值评分模式和传统综合评分模式对评分成绩的影响。结果显示,相对于综合评分模式,分步增值评分模式对评分结果的分布和区分度更好、评分者在评分偏差程度指标上的表现更加理想。虽然作文题评分依然普遍存在着评分等级不一致的问题,但采取分步增值评分模式可以提高评分成绩的合理性,并且更好地控制评分者评分成绩间的差异。此外,这种评分模式还能有效提高评分程序的效率。

The objectiveness and validity of many subjective performance rating assessments, especially in terms of writing assessment, remained problematic. It is necessary to introduce better rating procedures to reduce the errors of writing assessment. This current study applied muhifaceted rasch model（MFRM） to the rating scores of 500 national writing test papers to investigate the influences of the mul- tistage rating augmentation pattern to the ratings scores,as compared to the traditional holistic rating pattern. Results suggest the multi- stage rating augmentation pattern performed better in terms of level distribution and discrimination, and reduced the level of rating er- rors. Although the rating scores of writing tests were still prevalently inconsistent, multistage rating augmentation pattern could perhaps enhance the reasonableness of the assessment scores, and better control for the rating variations between different raters. Besides, multi- stage rating augmentation pattern may also enhance the efficiency in rating procedure of writing tests.