基于可达阵的多级评分最简完备Q矩阵设计

Design of the polytomous simplest complete Q matrix based on the reachability matrix

Q矩阵的完备性是认知诊断模型具有可识别性的关键。多级评分含有比0-1评分更丰富的诊断信息,却鲜见多级评分完备Q矩阵的设计研究。用最少的题量获得最高判准率是测验设计者追求的目标,借鉴0-1评分完备Q矩阵的设计方法,本文提出从可达阵中获取多级评分结构化/非结构化最简完备Q矩阵(SSCQM/USCQM)的方法和算法。模拟实验得出以下结论:(1)测验含SSCQM/USCQM越多,判准率越高;(2)当列数相同时,含多个SSCQM或多个USCQM测验的判准率与含可达阵测验的判准率非常接近;(3)对于一些结构,纵使多个SSCQM/USCQM的列数少于可达阵列数,其判准率仍不低于可达阵。总之,短测验设计优先选择SSCQM;长测验设计优先选择USCQM。

The identifiability of cognitive diagnosis models relies heavily on the completeness of the Q matrix.However,existing test designs primarily focus on dichotomously-scored items,neglecting the importance of polytomous cognitive diagnostic test design.Moreover,this limitation poses a significant obstacle to the advancement of cognitive diagnosis.To bridge this gap,this paper aimed to introduce novel designs for the construction of polytomous structured and unstructured simplest complete Q matrices(SSCQM/USCQM).Our proposed approach considered all ideal response patterns(IRPs)of knowledge states(KSs)on the reachability matrix as research objects,with the objective of minimizing the number of columns selected from the reachability matrix.This ensured one-to-one correspondence between the set of KSs and the set of IRPs,thereby enhancing the completeness of the SSCQM.Additionally,we derived a polytomous USCQM by considering the relationship between the SSCQM and the sub-matrix of the corresponding identity matrix while ensuring that each row contains at least one“1”.Interestingly,the construction process revealed that there were more USCQMs than SSCQMs.This innovative approach expanded the possibilities for polytomous cognitive diagnostic test design.This study focused on the design and evaluation of cognitive diagnostic tests using polytomous structured and unstructured Q matrices(SSCQM/USCQM).We conducted two studies to comprehensively examine the influence of factors such as the number of attributes,attribute hierarchies,and item parameters on the precision of the SSCQM,USCQM,and reachability matrix.In the first study,variations in attribute structures and item parameter values were investigated to understand their impact on Q matrix accuracy.On the other hand,the second study explored the effects of attribute hierarchies and the number of attributes on the precision of the SSCQM,USCQM,and reachability matrix.Both simulation studies and actual measurement data were utilized to assess the robustness and efficacy of the two methods.Firstly,the simulation results revealed several key findings.Notably,increasing the number of SSCQMs or USCQMs positively influenced the accuracy of the results.In the context of long tests,the USCQM demonstrated higher Pattern Match Ratio(PMR)and Marginal Match Ratio(MMR)compared to the SSCQM and the reachability matrix.This trend was particularly evident when there was an increase in item parameters,attribute numbers,or a change in attribute hierarchy.However,it is noteworthy that,regardless of these various factors,the PMR and MMR of the three tests exhibited minimal differences.On the other hand,in short tests with good item quality,the SSCQM achieved the best performance compared to other methods.This highlights the importance of considering specific test characteristics and item quality when selecting the appropriate Q matrix type.These findings provide valuable insights into the factors that influence the precision of Q matrices.They emphasize the benefits of increasing the number of matrices,understanding the impact of item parameters,and recognizing the performance disparities among different matrix types.Obtaining a comprehensive understanding of these relationships is vital for optimizing the design and implementation of cognitive diagnostic testing,ultimately guaranteeing accurate assessments of individual knowledge states.Secondly,analysis of the actual measurement data showed high identification repetition rates for the SSCQM and the reachability matrix,with a minimal difference in attribute mastery ratio.In summary,both the SSCQM and the USCQM demonstrate adequate performance when compared to other Q matrices under similar conditions.These findings emphasize the significance of prioritizing completeness in cognitive diagnostic testing.This research seeks to contribute to the advancement of cognitive diagnosis by addressing the limitations of existing test designs and introducing new techniques for constructing polytomous Q matrices.In addition,the findings presented in this paper offer valuable insights for researchers and practitioners seeking to design high-quality cognitive diagnostic tests that accurately assess individual knowledge states.