面试中的评分者贡献度指数及其应用

Rater Contribution Index and Its Application in Interviews

面试是过程评价和综合评价特别是高水平大学选拔拔尖创新人才的重要手段。但目前常用的面试质量评价方法,如评分者信度或概化系数估计方法,并不能快捷评估每个评分者的工作表现,影响了面试的质量。借助经典测验理论和概化理论,通过逐一核查各评分者评分信息缺失条件下的信度估计值变化情况,构造了一个评分者贡献度指数,并举例展示其使用方法和注意事项,为实时监控评分者的表现、保障和提高面试质量提供了新的方法。

Interviews are frequently used to evaluate a candidate’s competency during process evaluation and comprehensive evaluation,such as in college admissions and the recruitment process.However,currently used quality control methods such as estimating inter-rater reliability and generalizability coefficients do not provide details about each rater’s performance in an interview.Thus,a rater contribution index(RCI)is proposed based on classical testing theory and generalizability theory.The RCI for a specific rater is defined as the difference between overall raters’reliability estimate with that of all raters except the specific rater being evaluated.This paper discusses in great detail some examples of using RCI,as well as some words of caution.This should be very helpful for identifying unqualified raters in an interview.