基于多元概化理论的英语写作评分误差分析研究

A Study on Assessing Errors in ESL Writing Scoring: A Multivariate Generalizability Theory Approach

在英语测试中,如何对受试的作文做出较客观的评分是写作测试研究一直探索的问题。目前国内对英语写作测试评分人的评分行为及评分信度尚缺乏深入研究。本研究采用多元概化理论,通过实验发现,尽管写作样本的合成总分的信度较高,但评分标准中的各因子的评分信度有很大差异。评分员对评分量表中各写作因子的评分是否一致是造成评分误差的根源之一,应针对评分员在评分因子上的不一致加强评分人培训,以统一对评分标准的认识,建立一个客观、系统的作文评分系统。

It has long been recognized that the accurate scoring of English writing is notoriously difficult. Variability in writing scores associated with raters’ behavior is extensive, while in China empirical study in this field is not adequate. MGT approach is used to estimate the reliability of each scoring feature, aims to provide an in-depth exploitation into raters’ understanding and utilization of the scoring criteria. Analyses of these ratings showed that though high reliability could be achieved in composite universe scores, signif icant differences occur between the reliabilities of scoring features of the rating scale, which should be attributed to the difference of the raters in overall severity (or leniency) in certain feature. The study suggests that rater training should be strengthened based on their rating divarication in certain feature.