基于过程导向的英语写作评分量表效度验证

Process-oriented Validation of a Rating Scale for English Writing Assessment

本研究以概要写作评分量表为例,采用定量、定性相结合的融合型研究范式,运用多层面Rasch模型分析63名学生的概要写作得分,并结合七名评分员在使用量表过程中的有声思维和评分后的半结构式访谈回溯评分过程,探查评分量表在使用过程中的质量和作用,从而对该评分量表进行效度验证。结果表明:在评分过程中,评分量表能顺利引导评分员较好地把握评分宽严度和一致性。评分量表的描述语清晰准确,评分维度完整,有助于评分过程的顺利进行,但部分维度的分值权重应适当调整,评分员在各级别内部分值的选择上比较困难,需有针对性地改进评分量表。本研究对各类英语写作评分量表的效度验证具有一定的借鉴意义。

The present study adopts an integrated research pattern involving quantitative and qualitative methods to validate a rating scale for summary writing,and analyzes 63 students’scores of summary writing and results of think-aloud protocols and semi-structured interviews,using respectively Many-facet Rasch Model and text analysis in order to investigate the quality and effects of the rating scale in rating process.Results indicate that the rating scale can lead raters to score with appropriate leniency and rater agreement.The rating scale has clear and accurate descriptors as well as complete dimensions,which guarantee the smoothness of the rating process.The weighting,however,should be further adjusted.Raters has difficulty in discriminating scores within one level across all dimensions,indicating some changes accordingly.The study provides reference for the validation of other rating scales in English writing.