大规模开放在线学习学生互评效果实证研究

An Empirical Study on the Effect of Peer Assessment in Massive Open Online Learning

学生互评能够有效弥补教师评阅和机器评分的局限,是适用于大规模开放在线学习情境的重要评价模式。然而,现阶段对在线互评模式的准确性和有效性尚缺少基于实证的系统研究。基于此,本文通过对一门大规模开放在线课程(MOOC)的学生互评、自评和教师评分等数据的比较,得出在线互评模式的信度、效度、影响因素和学生认可度等初步结论。研究结果表明,尽管互评模式的评分者间信度并不理想,但综合考量多个评分结果的前提下,该模式能够为在线学习者提供较为一致可靠的最终得分。学生互评结果和教师评分结果的相关性系数高达0.619表明,该模式同时具有较好的聚合效度。此外,对课后问卷的统计分析表明,在线学习者对互评模式总体持积极态度,认可互评活动对反馈获得、课程投入和高阶思维培养等的有益影响。这些发现和结论对完善和改进面向大规模开放在线学习的评价模式有一定的指导意义。

Peer Assessment has the potential to overcome the limitations of teacher-grading and machine-grading,and therefore has become an important assessment method in the context of massive open online learning. However,there is a lack of empirical research that investigate the accuracy and effects of online peer assessment. To address such research need,this study analyzed the empirical data of peer assessment,teacher-grading and self-grading from a massive open online course（ MOOC）,and provided preliminary findings regarding the reliability,validity,influencing factors and perceived benefits of peer assessment for this specific context. The results show that although peer assessment suffers from inter-rater reliability,the composite score from all student graders can provide a reliable and consistent final result.The correlation coefficient of 0. 619 indicates a high convergent validity for peer assessment. In addition,this study also explored the influence of factors such as training and calibration,number of graders and score calculation methods. Moreover,the post-course survey results reveal that online learners in general have positive attitude towards peer assessment,acknowledging benefits such as feedback acquisition,higher engagement and development of higher-order thinking. The findings of this study are expected to inform the revision and improvement of assessment models for massive open online learning.