PETS口试评分培训效果的多面Rasch分析

Study on the effects of rater training for PETS speaking tests with Many-Facet Rasch Model

本研究以PETS-1级拟聘口试教师为研究对象,对口试教师评分的培训效果进行了研究。采用多面Rasch分析对比口试教师接受培训前后的评分效果。结果发现:培训后,提升了口试教师与专家评分完全一致的比率,评分偏于严格的口试教师在评分标准上做了恰当的调整,所有口试教师评分拟合值都在可接受范围内,总体上,口试教师评分的培训比较有效,培训后提升了评分的准确性。多面Rasch分析有助于发现评分过于宽松、过于严格、评分拟合差的口试教师以及评分异常情况,为开展有针对性的培训提供了可靠的依据。

This article describes a study conducted to explore rater training effects of the inexperienced raters on speaking tests, Public English Test System Level 1 （ PETS - 1 ）, by comparing the quality of scoring before and after training with Many- Facet Rasch Analysis. The analysis shows that, the raters achieve a significant increase in exact agreement percentage between inexperienced raters and expert judges after training. After training, too strict teacher ratings on speaking made appropriate adjustments, all teachers get the good Infit and Outfit values which are within the acceptable range. In general, rater training for PETS speaking tests reveals positive effects and ratings are improved. Many-Facet Rasch analysis helps discover too loose, too strict, and poor fitting raters, as well as abnormal scoring, which all provide a reliable basis for carrying out targeted training to raters.