新闻编译测试评分量表的研制与验证研究——基于多面RASCH模型的分析

Development and validation of scoring rubric for news trans-editing competence:A Rasch-based analysis

为了科学地量化考生在新闻编译测试中的作答表现,本研究基于新闻编译能力模型构建三分项五等级编译评分量表,包括内容转换的准确和客观、信息编辑的适度和合理、语言运用的准确与得体三大评分维度,另在文献分析的基础上编制新闻编译九级整体量表,并利用facet软件对3名评分员使用两份评分量表对同样60份编译文的评分结果进行对比分析,以验证量表的质量。数据显示,两份量表具有较好的信度和效度。相比之下,分项量表能更准确地区分考生的新闻编译能力,量表各分数段之间的区别更明显,评分员使用该量表时评分员效应更小,评分信度更高。

This study reports on the development of both an analytic and a holistic rating scale for the purpose of effectively measuring test-takers’English-Chinese News Trans-editing Competence(NTC).Following the construct of the NTC model,the paper proposes an analytic rating scale,which consists of three dimensions,namely content transferring,information editing,and language using.Each trait contains 5 bands.The holistic rating scale is adapted on the basis of Carroll’s 9-band holistic scale.The validity of the two scales was investigated by administrating a news trans-editing performance test to 60 examinees of different levels.Many-facet Rasch Model(MFRM)was later applied to analyze the scores awarded by 3 raters who adopted both the analytic and holistic scale on the same set of 60 news trans-edited versions.Statistics showed that(1)examinees were differentiated,but the analytic scales distinguished the subjects more accurately;(2)difference in rater severity was found among raters using the holistic scale,while there was no difference for analytic rating scale whose rater effects was smaller and inter-rater reliability higher;(3)the adjacent scales of analytic scale were more distinguishable.Generally speaking,the analytic rating scale performed better in measuring the NTC of candidates.