基于审题过程和考生成绩的学业考试效度检验

Exploring the validity of an achievement test based on test moderation and test scores

本文从审题过程和考生成绩角度来检验学业考试的效度。针对阅读学业考试,采用专家判断法和内省法收集审题过程中的数据,并使用Rasch模型分析考生成绩的数据。具体而言,阅读课程组三名教师和两名研究生助教参与本次的定性研究实验。一名教师从阅读课程配套材料中选取了一篇阅读材料及试题,另外两名教师和两名助教独立解答题目并记录解题思路。采用方便抽样,英语专业二年级75名学生在这篇阅读题目中的答题记录作为定量研究的数据。实验表明,学业考试能够较好地考核命题人员预期的认知技能,但是某些题型的考点设计尚有改进的余地。其次,试题的内容效度虽然比较理想,但是需要结合学生的实际水平不断打磨,减少偏易或者偏难的题目。

This paper reports on a validation study of reading achievement test based on test moderation and test scores. To this end, the data were gained by employing retrospective verbal reports by members of the reading course team and by conducting a Rasch analysis of examinees＇ test scores. Specifically, one instnaetor selected the reading comprehension items, then two instructors and two teaching assistants recorded their thinking-aloud processes in doing the items. Test records of 75 sophomores from the Department of English in a university were conveniently sampled to do the Rasch analysis. The results show that reading comprehension can not only elicit the expected reading opera- tions and expected reading performance to a large extent but also offer a good deal of evidence for content validity. As a whole, the samples of items and examinees were both reasonably fit for the Rasch model. However, certain test i- tems need to be moderated systematically, especially, some vocabulary items need to be revised due to their context irrelevance to examinees＇ responses.