话语填充测试方法的多层面Rasch模型分析

A many-facet analysis of Written Discourse Completion Test

本研究运用多层面Rasch模型从试题、考生、评卷人和评分量表等几个层面对话语填充测试方法在测试中国英语学习者的中介语语用能力中的表现进行了研究。197名中国大学生和24名英语本族语者参与了本研究。结果表明,两位评卷人在评分严厉程度上存在差异,但这种差异并不会影响最后结果的信度。试题的各个题目的难度有显著意义的区别,但是所有题目都和Rasch模型拟合。考生做题时表现出一定的差异,但总体来说他们的表现基本上是一致的。虽然本研究使用的评分量表有需改进的地方,但还是能够较好地区分考生的能力,基本上达到了预期的目的。

This study attempts to validate a Written Discourse Completion Test (WDCT) used in measuring the interlanguage pragmatic knowledge of Chinese EFL learners. A WDCT paper consisting of 12 request situations was carefully developed for this study. Development procedures included exemplar generation, likelihood investigation, metapragmatic assessment, and pilot study. A holistic rating rubric using a five?point Likert scale was developed and administered to 89 Chinese EFL learners. The WDCT papers were rated by two native speakers of English. Results show that the test items were significantly different in terms of difficulty and no item was found to be misfitting. The raters were found to have good internal self?consistency although they exhibited significant difference in severity. The probability curves for the 5 categories of the rating scale were shown to have distinct peaks and appeared as an evenly spaced series of hills. Generally, the rating scales of the rubric used in this study worked basically well for their purposes.