听力考试试题命制中的母语和外语效应

Effects of the use of second language and native language in formulating listening test questions

对于语言测试,使用母语和外语命制试题的效应意义重大。为了研究使用母语和外语命制听力试题的效应,我们邀请了来自中国大陆一所普通高校的234位大学本科生参加了一项英语听力理解测试。测试由3篇听力材料和10道四选一型听力理解试题构成。受试被随机分配到两个组,各组完成相同的听力任务,但测试的问题及其选项分别使用了英语和汉语。我们发现,命题语言因素对其中的有些问题有效应,对有些问题没有效应;选项中直接使用听力材料中的措辞对学生的听力成绩既可能产生积极影响,也可能产生消极影响;题干中的信息能引导学生关注听力材料中的有关信息;所用语言不同,相同意义干扰项的作用也可能不同。研究结果有多种启示:试题研发者可以使用干扰项所提供的信息来改进题目的质量,并需要不同语言的不同效应调整测试分数;如果是用于评价或诊断,测试使用者在解释结果时应考虑语言效应,并从干扰项数据中挖掘信息。

The effect of the use of the foreign language and the native language in formulating test questions is of great importance for language testing. To study this effect,we asked 234 Chinese undergraduates to do a short listening comprehension test of the 4-option type,including 10 items. The listening tasks are the same,but the questions are in either English or Chinese. We found language effects in some questions,but no effect in others. Direct matching of the expression in the choices with that in the recording may have positive or negative effect on students＇ performance; information in the stem can direct the students＇ attention to relevant information in the listening task; distracters may have different distractions when formulated in different languages. The results provide several suggestions. Test makers should use information provided by distracters to improve the item. They should consider the different language effect and adjust test score for such effect. When using the test results for program evaluation or diagnostic purposes,test users should consider the language effect in interpreting test results,and dig out information from the distracters data.