项目表述效应对自陈量表信效度的影响——以核心自我评价量表为例

The Impact of the Wording Effect on Reliability and Criterion Validity of Self-report Questionnaires——The Case of the Core Self-evaluation Scale

项目表述效应是指由项目表述方式的差异引起的与测量内容无关的系统变异,项目表述效应模型本质上是一种双因子模型。本研究以核心自我评价量表(CSES)为例,探讨项目表述效应对自陈测验信效度的影响。采用CSES、生活满意度量表和情感平衡量表对340名"蚁族"进行测查。结果表明,CSES在核心自我评价特质以外,还存在一个反向题项目表述效应因子;忽视项目表述效应对CSES的同质性信度和效标关联效度有重要影响:高估CSES的同质性信度,低估核心自我评价与生活满意度、积极情感的正相关,高估核心自我评价与消极情感的负相关。

When developing self-report questionnaires, researchers usually include negatively worded items to prevent response biases, such as acquiescence, affi rmation, and agreement bias. Yet this strategy that blends positively worded items with negatively worded items may lead to the method effect associated with the item wording（wording effect）, which might result in spurious variance in the relationships among the items. In research practice, however, researchers often ignore the potential wording effects： either directly construct a one-factor model, or calculate the total score of the whole scale as a manifest variable, then conduct a further study（e.g., the analysis of mediation or moderation effects）. This may not only inaccurately estimate the reliability, but also impact the correlations between the measured trait and other variables（e.g., the Criterion-Related Validity）, resulting in misleading conclusions. From the perspective of statistical analysis, the model with wording effects is a special bifactor model. The measured trait could be considered as the general factor, which explains the content variance among all the items, and the positive and negative wording effect could be accommodated by one or two group factors, accounting for the method variance introduced by the positively or negatively worded items. Under the orthogonality assumption between the general factor and group factors, the item response variance can be partitioned into the variance explained by the general factor（trait variance）, the variance explained by group factors（method variance） and the error variance, so researchers can compute the homogeneity coeffi cient and evaluate the impact of the wording effect on a unidimensional measurement model. Using the Core Self-Evaluation Scale（CSES） as an example, the current research aimed to examine the impact of wording effects on the reliability and validity of self-report questionnaires. A questionnaire including CSES, the Affect Balance Scale, and the Satisfaction With Life Scale（SWLS） was administered to 340 ＂Ant tribe＂. Two models were compared： one-factor model, and a bifactor model with a negative group factor. Results suggested that the bifactor model with a negative group factor fi tted better to the CSES data, that is, there was a method factor accounting for the method variance introduced by the negatively worded items over and above the general trait factor. The results showed that the impact of negative wording effects on the reliability and criteria-related validity of CSES was non-ignorable. The ignorance of negative wording effects might lead to considerable overestimation of the homogeneity reliability of CSES, overestimation of the correlation between the CSE and negative affect, but underestimation of both the positive correlation between the CSE and life satisfaction, and that between the CSE and positive affect. How to avoid and remove the wording effect？ When designing self-report scales, researchers should consider neutrally worded items, the valence of which is least extreme. Moreover, the use of bipolar items, instead of unipolar items, may get rid of the wording effects to some degree. When modeling, researchers should estimate both bifactor and unidimensional models, compare indicators＇ loadings on the general factor from the bifactor model with those from the unidimensional model, and report the homogeneity coeffi cient of the bifactor model. If indicators＇ loadings of these two models are compatible, and the homogeneity coeffi cient of the bifactor model is large enough（〉.7, say）, it is reasonable to report the total scores and use the unidimensional model for further analysis. Otherwise, it would be necessary to include the wording effect as a group factor in the model for unbiased estimates.