质性研究中编码者信度的多种方法考察

Testing Intercoder Reliability by Multi-approaches in Qualitative Research

质性研究中检验编码者信度的方法有归类一致性指数、编码信度系数、相关系数、中位数检验、概化系数等。基于教师胜任力访谈数据集,对编码者信度考察结果表明,归类一致性指数和编码信度系数受相同编码数影响而不稳定,相关系数受数据类型制约,中位数检验受研究设计影响,概化系数则受编码者和编码项目的数量影响。研究中须合理选用。

This paper reported several different statistical procedures for assessing intercoder （or interrater） reliability in qualitative research. The procedures included the index of category agreement, intercoder reliability coefficient, correlation coefficient, median tesst and G coefficient. Based on the data set of the interview study on teachers＇ competency, the results of analysis indicated that reliability indices obtained with different methods for coding in qualitative research had different efficiency. The index of category agreement and intercoder reliability coefficient was shown to be less stable and more affected by consistency among coders. Correlation coefficient was affeced by data type. Median test was limited by the research design. G coefficient was influenced by the number of items and coders. The authors suggested that researcher should choose the appropriate methods from these procedures and use them rationally.