概化理论在作文评分中的应用研究

APPLICATION OF GENERALIZABILITY THEORY IN COMPOSITION SCORING

概化理论是现代心理测量理论之一。该文简要地介绍了该理论的基本思想，并用此理论对作文评分的误差控制问题进行了应用性探讨。研究中请６位评分员对２０名学生每人三种文体的作文用分项评分法进行评定。然后用ＧＥＮＯＶＡ软件估计了作文评分中的评分员效应和题目效应，并对各种误差构成进行了分析比较。结果表明：在作文评分中，评分员效应最大，题目效应不明显。同时还发现，不同文体对评分误差有重要影响，议论文的评分误差最大。在作文能力的三要素中，以内容的评分误差最大，结构的评分误差最小。最后，本文还对传统评分者一致性系数（Ｗ系数）的不足以及概化理论中系数的优越性进行了讨论。

Generalizability Theory (GT) is a new trend in psychological measurement. First the basic ideas and methods of Generalizability Theory was introduced, Then an exploratory application of it was conducted on the problem of scoring in Chinese composition. Twenty high school students were asked to write three compositions of different forms in one month, all of the compositions were rated by six raters. Data was finally analysed by GENOVA. The results indicated that the influence of raters was large, while item effect was small It was also found that different forms of compositions had different scoring error, essay with problem discussion was likely to have the largest scoring error. As to the three essential factors of writing ability, the largest scoring error happened on the content, while the smallest scoring error happened on the structure. Furthermore, the author pointed out some defficiencies of the Kendall coefficient of concordance (W coefficient) and suggest that phi coefficient is a more effective index as rater consistency reliability.