PRETCO口试评分效度研究

A Study on the Scoring Validity of PRETCO-Oral

采用多层面Rasch模型,通过分析PRETCO口试的评分结果以探究其评分效度。研究发现PRETCO口试评分效度较高,其评分结果能够有效区分考生的口语水平,评分员评分的自身一致性总体较好。研究同时发现PRETCO口试评分存在以下问题:评分员的宽严度差别显著,个别评分员的内部一致性较差;少数评分员和考生的交互作用存在显著差异;评分员和四项任务之间也出现了不同程度的偏差。

This study explores the scoring validity of PRETCO-Oral through a many-facet Rasch analysis. Results show that the scoring of PRETCO-Oral is valid in that examinees＇ oral English proficiency can be screened by the test, raters＇ scorings are reliable and raters are self-consistent in general; However, results also demonstrate that there are significant differences in raters＇ leniency/severity; few raters exhibit self-inconsistency; there exists bias between several raters and examinees and bias between raters and the four tasks.