大规模英语口语考试评分质量保障体系建设——以CET-SET6在线评分为例

Construction of Rating Quality Assurance System for Large-scale English Oral Test:Taking CET-SET6 Online Scoring as an Example

大规模外语口语考试在线评分质量体系建设是一项系统工程,也是保障评分信度和可靠性的关键环节。在全国大学英语四、六级考试(CET)在线评分流程顶层设计指导下,三维度在线评分质量保障体系将口语评分过程中样卷及培训卷挑选流程、评分员培训流程、评分过程动态监控流程三个不同层面的子系统一环扣一环关联起来,使标准化工作机制一步一步落到实处。该体系从宏观和微观的管理视角,自上而下而又自下而上地从不同层面对评分误差进行控制,以降低评分中因评分员主观因素造成的给分差异。CET-SET6在线评分实践表明,该保障体系的实施提升了CET-SET评分信度和效度,有效保障了评分质量。该体系的建设也可为提高大规模、高风险考试在线评分信度、效度及质量管理水平提供一定的借鉴和参考。

Establishing a quality assurance system for online scoring of large-scale foreign language oral proficiency examinations is not only a comprehensive undertaking but also a crucial step in ensuring the reliability and validity of scoring.Guided by the overarching framework of the process for the College English Test(CET)in China,the development of a three-dimensional quality assurance system integrates various subsystems at different levels to ensure the selection of range finders and samples for rater training assessment,the online training of raters,and the dynamic monitoring of the scoring process.This systematic approach facilitates the step-by-step implementation of standardized mechanisms.From both macro and micro management perspectives,this system effectively controls scoring errors at different levels,thereby reducing scoring discrepancies resulting from subjective factors among raters.Practical application of this quality assurance system in CET-SET6 online rating demonstrates its effectiveness in enhancing the reliability and validity of scoring,consequently ensuring the overall quality of the scoring process.Furthermore,the construction of this system can serve as a model for enhancing the management of online rating quality in other large-scale,high-risk oral test scoring.