机考口试评分方式对比研究

A comparative study of computer-based speaking test scoring methods

分项评分和任务评分是常见的两种机考口试评分方式。本研究采用多面Rasch分析和概化分析等多种数据统计方法,从评分员严厉程度、评分标准使用、考生能力区分等多个角度对两种评分方式进行综合对比。研究结果表明,两种评分方式在评分员、评分标准、考生等层面显示出一些共同特点,比如评分员自身一致性较好但评分员之间严厉程度不一致、评分存在趋中现象等。在对考生整体口语能力的区分和测试的准确程度方面,分项评分均优于任务评分。

Analytic scoring and part scoring are two common scoring methods of computer-based tests.From the perspectives of harshness of raters,use of rating scales and distinction of test-takers abilities,this study applies multiple statistical analyses such as multi-facet Rasch analysis and generabilizability study to conduct a comparative study of the two scoring methods.The study shows that the two methods demonstrate common features from the facets of raters,scales,and test-takers,such as the high intra-rater consistencies but low inter-rater consistencies,and central tendency of scores.In terms of the distinction of test-takers’overall speaking abilities and accuracy of test results,analytic scoring performs better than part scoring.