翻译测试的评分策略研究——基于CET-4翻译任务

An Investigation of Rating Strategies in the Translation Assessment: Based on CET-4

采用混合研究法探究CET-4翻译评分员使用的评分策略:首先,邀请13位评分员对5篇CET-4翻译作答进行有声思维评分,据此生成翻译评分策略问卷初稿;然后,在对问卷初稿进行试测后对255位评分员施测;最后,对问卷的信度、效度进行检验并删减题项,得到问卷终稿。结果表明,最终问卷包括19个题项,整体信度为0.886,翻译评分策略结构包括"修改""诊断""自我评估""推测"和"比较"5个因子,可解释总方差的58.249%。本研究可为大规模语言测试翻译题的设计和评分提供参考。

Research on rater cognition plays a critical role in providing evidential support for a validity argument for scores of constructed tasks. However, there is little empirical evidence pertaining to how raters make scoring decisions for the translation task. To this end, this paper aims to investigate CET-4 translation raters’ rating strategies following the mixed methods design. First, thirteen raters were invited to rate five CET-4 translation mock scripts while thinking aloud in order to generate the initial version of translation rating strategy questionnaire. After piloted on three raters, it was administered to a sample of 255 raters. Finally, item analysis, reliability analysis and exploratory factor analysis were conducted to check its psychometric properties. It was found that the final version of the instrument contains 19 items;the overall reliability reached 0.886 measured by Cronbach’s Alpha;and the fivefactor solution(revise, diagnose, self-evaluate, infer and compare)could explain 58.249% of the total variance.Implications are discussed in terms of design features of translation tasks and rater training in large scale language tests.