联合作答精度和作答时间的概率态认知诊断模型

Joint cognitive diagnostic modeling for probabilistic attributes incorporating item responses and response times

对多模态数据的联合分析是改进结果评价、健全综合评价的主要途径。针对概率态认知诊断模型(CDM)仅能分析题目作答精度(RA)的局限,本文基于联合-层级建模框架和联合-交叉负载建模框架提出三个可联合分析RA和题目作答时间(RT)的概率态联合CDM。模拟研究和实证研究结果表明:(1)新模型参数估计返真性良好,额外引入RT有助于提高参数估计精度并提供有关个体加工速度的测量;(2)基于联合-交叉负载建模框架构建的模型对测验情境的兼容性优于基于联合-层级建模框架构建的模型;(3)概率态属性比确定态属性更精细地反映个体对属性的掌握情况。

Compared with the conventional CDM with deterministic or binary attributes,the CDM with probabilistic attributes(probabilistic-CDM)can achieve a more refined diagnosis of attribute mastery status,which helps distinguish individual differences between students and provides more reference information for teacher feedback.However,existing probabilistic CDMs can only analyze a single modal of data—item response accuracy(RA),ignoring other modals of data such as item response times(RTs).RTs reflect the cognitive processing speed of the participant.With the popularity of computerized testing,recording RT data has become routine.However,how to use RTs in probabilistic CDM to further improve parameter estimation accuracy and enrich the diagnostic feedback information is still an unsolved methodological problem.To this end,the current study proposes three joint probabilistic CDMs based on the joint-hierarchical and joint-cross-loading cognitive diagnostic modeling approaches.First,based on joint-hierarchical modeling,the joint-hierarchical probabilistic CDM(JRT-PINC)was proposed in Study 1,which achieved the purpose of using RT to improve diagnostic accuracy.A simulation study was conducted to investigate the psychometric performance of the JRT-PINC under various simulated testing conditions,in which three independent variables,including sample size,test length,and the correlation between person parameters,were manipulated.Second,two joint-cross-loading probabilistic CDMs(CJRT-PINC-θand CJRT-PINC-m)were proposed based on the joint-cross-loading modeling.In contrast to the JRT-PINC model,two CJRT-PINC models directly used RTs to provide information for latent abilities or attributes by introducing item-level cross-loading parameters.Two CJRT-PINC models released some conditional independence assumptions in JRT-PINC,increasing their application scope.Two simulation studies were conducted to explore their performance under different simulated conditions with different degrees of cross-loading.Third,Study 3 aims to explore the relative merits of the JRT-PINC and two CJRT-PINC models,that is,the necessity of considering cross-loading in the joint analysis of RA and RT.Finally,an empirical example was conducted to illustrate the practical applicability of the proposed models and to compare them with existing CDMs(e.g.,CDMs with deterministic attributes).The simulation results mainly indicated that:(1)all three proposed models can be well recovered under different simulated conditions;(2)CJRT-PINC-θmakes fuller use of the information contained in RTs and thus improves the accuracy of the parameter estimation of the core constructs(e.g.,latent ability and attributes)than CJRT-PINC-m;and(3)the adverse effects of ignoring the possible cross-loadings are more severe than redundantly considering them.The results of the empirical example indicated that:(1)probabilistic attributes provide more refined feedback on participants'mastery of attributes than deterministic attributes;and(2)two CJRT-PINC models fit this data better than the JRT-PINC model.Overall,this paper introduced RTs in probabilistic CDM for the first time and proposed three joint probabilistic CDMs based on two joint cognitive diagnostic modeling approaches.This study enriched the scope of application of probabilistic CDMS and provided methodological guidance for further refined and comprehensive diagnosis by jointly analyzing multi-modal data in technology-enhanced assessment systems.