认知诊断模型中被试参数估计方法的选择

The Choice of Examinee Parameter Estimation Methods inCognitive Diagnostic Models

根据被试的反应数据获得精细化诊断信息,为研究者提供个性化的指导是CDM的主要目的之一。以往研究在DINA模型下比较了被试参数估计方法(MLE、MAP和EAP)的表现,但在实践中如何选择最有效的方法仍有待研究。本文从理论角度探讨了不同知识状态分布对被试参数估计方法的影响,并进行了模拟研究。结果发现:当属性之间相关时,EAP和MAP方法的分类结果相似并高于MLE。鉴于实践中属性一般呈中等或高相关,建议选择EAP/MAP作为被试参数估计方法。

Cognitive diagnostic models(CDMs),which are also referred to diagnostic classification models(Rupp et al.,2010),are multiple discrete latent-variable models.In the past few decades or even earlier,CDMs have become a popular method in many fields,such as psychological and educational measurement,psychiatric evaluation,and other disciplines.Arguably,to offer fine-grained differentiated diagnostic information based on the examinees'observed response data to further help teachers and clinicians taking individualized instructions or interventions is one of the ultimate purposes of CDMs.Three examinee parameter estimation methods have been proposed to classify examinees into a group of latent classes in CDMs,including the maximum likelihood estimation(MLE;Birnbaum,1968),maximum a posteriori(MAP;Samejima,1969)and expected a posteriori(EAP;Bock&Mislevy,1982).Huebner and Wang(2011)investigated the performance of MLE,MAP,and EAP for classifying examinees within the DINA model framework.They found that MLE/MAP had a higher correct classification rate on all K skills.In their study,however,the item parameters and structural parameters were assumed to be known.Although the previous study compared the performance of the MLE,MAP and EAP,the choice of the most suitable examinee parameter estimation methods in CDMs still tend to be a problem.In this study,we proposed that the main difference between MLE,MAP and EAP is that the last two methods consider the latent knowledge state distribution.Thus,a simulation study was conducted to investigate the impact of latent knowledge state distribution on the classification accuracy of MLE,MAP and EAP.Five factors were manipulated:the attribute tetrachoric correlation(0,.5 and.8),number of sample size(300,1,000 and 5,000),number of attributes(3 and 5),data-generated models(DINA,DINO,A-CDM and G-DINA)and the types of Q-matrices(correctly and incorrectly).Four evaluation criteria were pattern correct classification rate(PCCR),attribute correct classification rate(ACCR),the classification rate for each skill(Skillk)and the average of the classification rate for all skill(Total).The classification results for all four criteria were averaged over the 1000 replications.Results showed that,(1)When the attribute tetrachoric correlation was zero,MLE produced the highest correct classification rate with the criteria of PCCR;(2)When the attribute tetrachoric correlation was moderate or high,the EAP and MAP generally yielded higher classification rate than that of the MLE;(3)The correct classification rate increased as the attribute correlation and item quality increased;(4)The correct classification rate of the misspecification of Q-matrix were worse than those in true Q-matrix and items with more attributes had lower accuracy;(5)The DINA and DINO models yielded more accurate classification rate than the G-DINA and A-CDM models.Overall,choosing the most appropriate knowledge state estimation method is of theoretical and practical importance.The results of this study indicated that the classification accuracy of MLE,MAP and EAP were affected by the latent knowledge state distribution,we recommend using EAP/MAP as an estimation method in practice to ensure the accuracy of estimation.