表现性评价中评分者信度估计方法的比较研究——从相关法、百分比法到概化理论

A Comparative Study on Methods Used in Estimating the Inter-rater Reliability of Performance Assessment——Methods Based on Correlation, Percentage of Inter-rater Reliability and the Generalizability Theory

随着素质教育的推进,表现性评价受到越来越多的重视。影响表现性评价结果信度的一个重要因素是评分者之间的不一致。文章使用模拟数据,在对比评分者一致性的相关法、一致性百分比法和概化系数等各种估计方法的基础上,提出概化理论在表现性评价中评分者信度问题上的应用是理论和实践发展的有益方向。

As quality-oriented education presses on, performance assessment is attracting more and more attention. One major source of unreliability in performance assessment is inter-rater disagreement. By comparing results from different methods, including bivariate correlation, intraclass correlation, percentage of inter-rater reliability and the generalizability theory, we highlighted the advantages and disadvantages of each method and the generalizability theory in estimating the inter-rater reliability of performance assessment.