英语机考的信效度研究

Reliability and Validity Study of A Computer-based English Test

文章以PETS-5为例,在设计并实现机考形式后,选取出国培训班学生共404人参加了该机考测试。测试结束后,通过在线问卷调查的形式调查了被试对该机考形式的感受与意见,并收集了考生参加PETS-5纸笔考试的考试成绩。研究结果表明:(1)该机考总体的Alpha信度为0.82,各部分为0.63~0.65,具有较好的内部一致性信度;(2)该机考各部分的相关系数为0.34~0.43,与纸笔考试各部分相关的结果基本相近,具有较好的结构效度;(3)该机考成绩与纸笔考试成绩的相关系数为0.709,具有较好的效标关联效度;(4)该机考模式对于男生和女生是公平的,试题在性别上不存在项目功能差异;(5)关于测验总分及各部分性别差异t检验结果显示,男女考生仅在阅读理解成绩上存在性别差异,男生的平均成绩高于女生。

In this study, PETS-5 computer-based English test is designed and implemented. A total of 404 students from overseas training class are selected to take the test. After the test, a questionnaire survey is conducted online to investigate the feelings and opinions of the participants towards the computer-based test. And the PETS-5 paper-pencil test scores of the examinees are collected. The results show that （1） the overall Alpha reliability of the computer-based test is 0.82, and the reliability of every section is 0.63- 0.65, which indicate good internal consistency reliability. （2） The correlation among various sections is 0.34-0.43, similar to the results of each part of the paper-pencil test, which reveals good construct validity. （3） The correlation between the computer-based test scores and paper-pencil test scores is 0.709, which means good criterion-related validity. （4） The computer-based test is fair for males and females, and there is no gender DIF in the test. （5） The t-test results show that male and female candidates only have gender differences in reading comprehension scores, and the average score of the male is higher than that of the female.