口语评分中评分员对评分标准的理解和使用——配对口试评分的报告分析

Raters' Understanding and Utilization of the Rating Scale in an Oral Test:An Analysis of Scoring Process in a Paired Candidate Test

通过有声思维实验方法并辅以刺激回忆,收集四名不同性格倾向的评分员在配对口语考试评分时进行的思维报告数据,定性分析结果表明:在实际评分中,评分员对评分量表的理解和使用存在很大的差异性,具体表现在:(1)外向的评分员在评分过程中,表现的比内向的评分员更为宽容;(2)内向的评分员更多地关注评分量表中的各项具体指标和标准,而外向的评分员强调任务的完成状况和考生之间的比较、交流,和互动;(3)外向的评分员比内向的评分员更少地依赖评分量表,更多地使用非语言的特征。本研究结果对考试评分标准的修订和评分员培训均有启示。

This study adopted Think Aloud Protocol and stimulated recall to collect thinking data of four raters with different personality types. Qualitative researches demonstrate that raters have distinct understanding and utilization of rating scales. The more detailed findings are： （1）the introverted raters are more severe than extroverted ones; （2）the introverted raters pay more attention to the specific features of the rating scale, whereas extroverted raters concentrate task realization and the interaction of candidates ; （3）the extroverted raters attend to more non - criterion features, and rely much less on the rating scale. These findings have implications for both the development of rating scales and the training of raters for paired candidate discussion tasks.