托幼机构教育质量评价中评委偏差的多侧面Rasch分析

Many-Facet Rasch Model Analysis of Rater Bias in the Measurement Quality of the Early Childhood Program

采用多侧面Rasch模型对28位评委在托幼机构教育质量评价中的评委偏差进行了分析。分析结果显示:28名评委评分宽严度差异显著;3名评委内部一致性较差,其余25名评委内部一致性较稳定;评委与评价班级的交互作用不显著,与评价项目的交互作用显著。研究结果表明MFRM可以对托幼机构教育质量评价的评委偏差进行个体层面的具体分析,从项目反应理论的视角为托幼机构教育质量评价的评委针对性培训、评估评委的合格性从而建立合格评委库等提供现代教育、心理测量学依据。

The assessment of early childhood programs is of great importance for children’s development and education. It is often assessed by individuals subjectively, while the bias from the subjectivity of the individuals tends to be neglected. We hold that it is necessary to analyze the bias from the variable conditions of the assessments.155 classrooms were sampled from the eastern part of China, and they were observed and measured by 28 raters. The raters were college teachers and postgraduate students specialized in preschool education, and they were all trained, and had had experience, to assess the quality of early childhood programs before. The scale in the study is called the Chinese Early Childhood Environment Rating Scale（CECERS）, which is made up of 46 items, 8 subscales. Usually, one classroom was observed and assessed by 2 or 3 raters at the same time. In this measurement situation, rater severity, inter-rater reliability, the bias between the rater and the class, and the bias between the rater and the item, were analyzed from three facets. The three facets were the class, the rater and the item respectively. Facets software（MFRM3.68.1） and SPSS were adopted in this study. The results of the analysis revealed that： there was significantly difference among the 28 raters; the consistency from the 3 raters was high, at the same time, the internal consistency was stable from the rest of 25 raters; a further statistics of the unexpected responses from the three raters showed that, there were no common attributes among the classes and items, in other words, the unexpected responses were not fixed on the same kind of classes or items, the interaction between the rater and classroom was not different significantly, the ratio of the significant bias was 8.02%, the further analysis based on the significant bias showed that, there were no common attributes among these classes, which is to say, the kinds of the biased classes were various, the ratio of the significant bias between the rater and the item was 8.2%, but there was no significant difference during the interaction between the rater and item. The items with significant bias was computed further, it was found that there was a large deviation in the item 45 and item 13;the deviation was less in the items such as item 22, 25 and so on, which were more familiar for the raters.All of these findings may provide the psychometric basis for the training, the standards and the construction of the rater bank from the perspective of IRT.