大规模英语考试作文评分标准效度验证

Validating TEM-8 Rating Scale via MFRM and TAPs Based Evidence

本研究收集评分员对130篇大学专业英语八级考试(TEM-8)作文的评分数据,采用多面Rasch模型分析法以及有声思维法收集证据对TEM-8作文评分标准进行了多维度效度验证。结果表明,评分标准大体上能够反映写作理论构念,评分尺度划分较为合理;大部分评分员能够有效使用评分标准进行评分,可信度较高。

The validation study of the rating scale used in Test of English Majors（TEM-8）writing assessment was conducted based on data from the rating scores of 13 raters each rating 10 TEM-8 essays and their Think-aloud Protocols during rating. Quantitative analysis using Multi-Facets Rasch Model and qualitative analysis using Nvivio showed the following results： 1）the rating scale is appropriately categorized and operates well during the rating process; 2）although a few raters＇ ratings exhibit some misfit and severity, the majority of the raters are able to rate with high consistency using the scale.