网络测验和纸笔测验的测量不变性研究——以生活满意度量表为例

Examining the Measurement Invariance between Paper-and-Pencil and Internet-Administered Tests in China

以生活满意度量表为例,运用实证性因素分析,考察在中国文化下网络测验和传统纸笔测验之间的测量不变性。结果显示,网络测验和纸笔测验之间存在弱不变性,即网络测验和纸笔测验有着相同的测量单位;但网络测验和纸笔测验只存在部分的强不变性和部分的严格不变性,测验实施环境对结果的影响不可忽视。该研究表明,恰当设计的网络测验是可靠的,同时还提示,当一个测验在不同情境下运用时,检验测量不变性十分必要。

Concerns about the quality of internet-based tests have been brought by the increasing applications of such tests in psychological research. Over the past years, a large body of studies has been conducted to examine the equivalence of intemet-administered tests to their paper-and-pencil counterparts. Although studies based on Classic Test Theory （CTT） showed that internet-administered tests were trustable, studies based on measurement invariance tests produced mixed findings. What is more, most studies so far have been conducted in individualistic cultures. Given these,, the present study aimed to examine the equivalence between internet-based and paper-and-pencil tests in a collectivistic culture, particularly, in China. To this end, we employed Confirmatory Factor Analysis （CFA） to examine the measurement invariance of the selected scale： Satisfaction with Life Scale （SWLS） across modes. SWLS was administered via intemet and the paper-and-pencil modes. Five items were rated on a 7-point likert scale ranging from 1 （＂strongly disagree＂） to 7 （＂strongly agree＂）. A total of 418 serf-selected college students from 26 provinces in China took the intemet-based test. And a total of 288 college students at Sun Yat-Sen University were sampled to take the paper-and-pencil test in classroom. For the intemet sample, the age ranges from 18 to 24 years old with a mean of 21.31 （SD = 1.38）; for the paper-and-pencil sample, the age ranges from 18 to 24 years old with a mean of 2020 （SD =0.81）. Multi-group CFA was employed to test measurement invariance between the Internet aommlstered ano the paper-and-pencil SWLS. Results showed weak measurement invariance held across these two test modes, indicating metric similarity between the tests; partial strong measurement invarianoe and partial strict measurement invariance also held, suggesting that response bias existed in some items across modes; further analysis revealed that the paper-and-pencil test included more noise arising from administering environment. In terms of mean comparisons, significant differences between modes were found in observed scores but not in latent scores. For the variances, no significant differences were found between modes in either latent scores or observed scores, These findings suggested that administering environments produced potential impacts on observed scores. As the first examination of the measurement invariance in Chinese samples, the study provided initial evidence that internet-based tests have equivalent metrics with paper-and-pencil tests. Further, the results from the partial strong invariance and partial strict invariance may indicate the sensitivity of Chinese people to environments that may be resulted from collectivistic culture. Taken together, the findings from this study suggest that although intemet-based tests are trustable in China, cautions of response biases should be kept in mind when conducting cross-groups （or modes） comparisons. Also, the findings underscore the importance of examining measurement invariance when a test is applied across multi-groups （or modes）.