基于CD-CAT的多策略RRUM模型及其选题方法开发

Item Selection Using the Multiple-Strategy RRUM Based on CD-CAT

在有多种解题策略的认知诊断问题情境中,用每个Q矩阵表示一种解题策略,由此将单策略认知诊断RRUM模型拓广为多策略RRUM模型(MS-RRUM)。随后,在应用MS-RRUM模型的CD-CAT中开发了适用于多策略情境的MAP参数估计法和多策略香农熵(MSSHE)选题法。将MSSHE选题法与随机选题法分别在不同属性数量、不同测验长度下进行比较,结果发现前者对被试的策略和认知状态判准率都显著优于后者,而且都很理想。这样就顺利实现了在CD-CAT做策略诊断的目标。

Cognitive diagnostic computerized adaptive testing （CD-CAT） is an application of cognitive diagnostic testing （CDT） on the computerized adaptive testing （CAT） platform. CD-CAT can provide information on the specific content areas in which a participant needs the most specific help, and it is typically engineered to tailor the test to each participant＇s trait level and thus has high efficiency. The search for the best item selection method is currently one of the most pressing issues in the field of CD-CAT research. Besides, studies on CD-CAT have to address the issue on the number of testing strategies to employ： either all participants use the same strategy, or different participants use different strategies. In practice, for some tasks, all participants use the same strategy; whereas for other tasks, different participants use different strategies to solve the problem. Because cognitive strategy aptitudes （or cognitive structures） vary among participants, each strategy requires different attributes. For the latter type of tasks, a participant＇s strategy must be identified before his/her knowledge state （KS, i.e., attribute mastery pattern） can be diagnosed. Strategy diagnosis procedures have not been explored in studies on CD-CAT. Based on the reduced reparameterized unified model （RRUM） and independence attributes, under different lengths of fixed-length CD-CAT and with various numbers of knowledge attributes, using the maximum a posteriori （MAP）, the present study explored the use of the multiple-strategy RRUM （MS-RRUM） and the item selection method in the context of multiple strategies. The MS-RRUM extended the traditional single-strategy RRUM by constructing M different Q-matrices. In the MS-RRUM, each strategy for each item involved a set of parameters, including one baseline parameter and several penalty parameters. The MAP method was then generalized to the multiple-strategy context. An item selection method that fitted the multiple-strategy context, named as the multiple-strategy SHE （MSSHE）, was proposed here. When a participant responded to an item, the system estimated the participant＇s strategy according to his/her item response vector and then estimated the participant＇s KS using the selected strategy. The next item was then selected via the MSSHE method. Using the MS-RRUM along with varying numbers of attributes and test lengths, the strategy and KS recoveries of the MSSHE and the random method were systematically compared. Results showed that the MSSHE method exhibited excellent strategy and KS estimations, and it was superior to the random method. Besides, when the test was short, the advantage of the MSSHE was more obvious in both strategy estimation and KS estimation; when the number of attributes was relative large, the advantage of the MSSHE was more obvious in KS estimation. Therefore, （1） the MS-RRUM is feasible and can be used in the CD-CAT context; （2） the MAP method, when extended to the multiple-strategy context, can successfully estimate participants＇ strategies and KSs; （3） when CD-CAT is applied to the MS-RRUM, the proposed MSSHE method is feasible and efficient in item selection. This study is the first to apply CD-CAT in a multiple-strategy context, providing more sufficient diagnostic information. Furthermore, the proposed method of MAP in the multiple-strategy context makes parameter estimation easier and is time-saving, and the proposed MSSHE method makes item selection efficient and saves item bank. Overall, this study serves to facilitate the implantation of multiple strategies in CD-CAT. Further studies are suggested to examine the performance of these approaches under other conditions （i.e., using other cognitive diagnostic models, attribute hierarchical structures and variable-length CD-CAT settings）.