CD-CAT中基于SCAD惩罚和EM视角的在线标定方法开发——G-DINA模型

Development of Online Calibration Method based on SCAD penalty and EM perspective in CD-CAT:G-DINA model

G-DINA(the generalizeddeterministic input,noisy and gate)模型限制条件少,应用范围广,满足大量心理与教育评估测验数据的要求。研究提出一种适用于G-DINA等模型的同时标定新题Q矩阵与项目参数的认知诊断计算机化自适应测验(CD-CAT)在线标定新方法SCADOCM,以期促进CD-CAT在实践中的推广与应用。本研究分别基于模拟题库以及真实题库进行研究,结果表明:相比传统的SIE方法,SCADOCM在各实验条件下均具有较为理想的标定精度与标定效率,应用前景较好;SIE方法不适用于饱和的G-DINA等模型,其各实验条件下的Q矩阵标定精度均较低。

Cognitive diagnostic computerized adaptive testing(CD-CAT)provides a detailed diagnosis of an examinee’s strengths and weaknesses in the content measured in a timely and accurate manner,which can be used as a reference for further study or remediation planning,thus meeting the practical need for efficient and detailed test results.The successful implementation of CD-CAT is based on an item bank,but its maintenance is a very challenging task.A psychometrically popular choice for maintaining an item bank is online calibration.Currently,the research on online calibration methods in the CD-CAT that can calibrate Q-matrix and item parameters simultaneously is very weak.The existing methods are basically developed based on the deterministic input,noisy and gate(DINA)model.Compared with the DINA model,the generalized DINA(G-DINA)model has been more widely applied because it is less restrictive and can meet the requirements of a large number of test data in psychological and educational assessment.Therefore,if the online calibration method that jointly calibrates the Q-matrix and item parameters can be developed for models with few constraints such as G-DINA,its meaning is understood without explanation.In current study,a new online calibration method,SCADOCM,was proposed,which was suitable for the G-DINA model.The construction of SCADOCM was based on the smoothly clipped absolute deviation penalty(SCAD)and marginalized maximum likelihood estimation(MMLE/EM)algorithm.For the new item j,the log-likelihood function with SCAD can be formulated based on the examinees’responses in this item and the examinees’attribute marginal mastery probability,and the q-vector of the new item can be estimated by the q-vector estimator based on SCAD.Then,the EM algorithm was used to estimate the item parameter of the new item j based on the posterior distributions of examinees’attribute patterns,the examinees’responses to new item j and the estimated q-vector.To examine the performance of the proposed SCADOCM and compare it with the SIE method,two simulation studies(Study 1 and Study 2)are conducted.Study 1 is based on a simulated item bank while Study 2 is based on the real item bank(Internet addiction item bank;Shi,2017).In these simulation studies,four factors were manipulated:the calibration sample size(nj=50 vs.100 vs.500 vs.1000 vs.2000),the distribution of the attribute pattern(uniform distribution vs.high-order distribution vs.normal distribution),the item quality(U(0.05,0.15)vs.U(0.1,0.3)),and the online calibration methods(SCADOCM vs.SIE).The results showed that(1)SCADOCM has satisfactory calibration accuracy and calibration efficiency,and is superior to the SIE method.In addition,the traditional SIE method is not applicable for the G-DINA model,and its Q-matrix estimation accuracy rate is low under all experimental conditions.(2)The item calibration accuracy of SCADOCM and SIE increases with the increase of calibration sample and item quality under most conditions,and its item calibration accuracy in the uniform distribution/higher-order distribution is greater than that in the normal distribution.(3)The calibration efficiency of SCADOCM decreases with the increase of calibration samples,but it is less affected by the item quality and the attribute pattern distribution;the calibration efficiency of SIE decreases with the increase of calibration samples,but it is less affected by the item quality.Moreover,the calibration efficiency of the SIE method in the normal distribution is slightly slower than that of uniform distribution/high-order distribution.To sum up the results,this study demonstrated that the SCADOCM has higher item calibration accuracy and calibration efficiency,and outperforms the SIE method;meanwhile,the traditional SIE method is not suitable for G-DINA model.All in all,this study provides an efficient and accurate method for item calibration in CD-CAT,and provides important support for further promoting the application of CD-CAT in practice.