慕课学生互评误差纠正方法及其比较

A Comparison of Methods to Correct Errors in Peer Assessment Ratings in Massive Open Online Courses

学生互评是广泛用于慕课的一种评价方法,然而学生评估者本身存在比较大的评分误差。本文着重介绍和比较可用于纠正慕课学生互评误差的方法。这些方法总体分为两大类,即对学生评估者进行前期纠正的方法和对评分结果进行后期纠正的方法。文中总结的绝大部分方法目前都还没有被实际运用在慕课学生互评中。希望通过本文对慕课学生互评以及纠正学生评分误差方法的介绍,可以让更多的教育研究者参与对慕课的评价系统进行改善的研究。

Peer assessment is one of the most important assessment methods in Massive Open Online Courses（MOOCs）, especially for open-ended assignments or projects. However, for the purpose of summative evaluation,peer assessment results are generally not trusted. This is because peer raters, who are novices, would produce more random errors and systematic biases in ratings than would expert raters, due to peer raters＇ lack of content expertise and rating experience. In this paper, two major approaches that are designed to improve the accuracy of peer assessment results are reviewed and compared. The first approach is designed to calibrate accuracy of individual peer raters before actual peer assessments so that differential weights can be assigned to raters based on accuracy.The second approach is designed to remedy peer rating errors post hoc. Differences in assumptions,parameterization and estimation methods, and implementation issues are discussed. The development of methods to improve MOOCs peer assessment results is still in its infancy. Most of the methods reviewed in this paper have yet to be implemented and evaluated in real- life applications. We hope the discussion and comparison of different methods in this paper will provide some theoretical and methodological background for further research into MOOC peer assessment.