基于群体水平评估的认知诊断模型开发与应用

Developing A Group-level Cognitive Diagnosis Model and Its application

开发了基于群体水平评估的认知诊断模型——G-AHM,用Monte Carlo模拟探讨模型的性能与表现及其在实践中的具体应用。研究发现:(1)G-AHM具有较高的边际判准率、较好的模式判准率和较强的稳健性,说明新模型基本合理可行。(2)与已有较好的认知诊断模型比较发现:从认知状态、属性掌握概率与属性掌握比例三个方面,G-AHM模型所获得的群体诊断结果都与已有结果基本一致,即可认为G-AHM具有较高的效度。特别地,G-AHM将认知状态与群体对属性的掌握概率信息相结合,可更好的解释及分析被试的认知水平,提供的信息更具参考价值。

Group assessments were aiways an important field, buGroup assessments were always an important field, but almost all of the developed cognitive diagnosis （CD） models could only realize the individual-level CD assessment. The group-level CD assessment was performed by the attribute mastery percent based on individual-level CD assessment. But this method was time consuming and demanding. This paper tried to develop a new model called the Group-level Attribute Hierarchy Model （ G-AHM）, which could realize the group-level CD assessment directly. This model could not only provide the group knowledge states information, but also the attribute mastery probability （AMP） information. When there was only one person to answer per item, the G-AHM became the RSM or AHM model corresponding to the decision method. To investigate the properties of the model, two studies were done. One was the Monte Carlo simulation study, which was to detect the rationality and feasibility of the model. The other was a real data study on the solution test of English reading problem. In this study, the G-AHM was utilized to realize the group-level CD assessment. And a comparison was done between it and another research, the effective and corrected matched ratio of which were relatively great. Thus, the feasibility and application of the model in real work could be discussed. The findings showed that ： （ 1 ） The simulation study showed that the new model was rational and feasible. The three types of decision method supposed in the model have different effects on the group-level CD assessment. The Bayesian decision method （BDM） was the worst, which was the same finding from the other studies. The corrected decision ratio of the model was relatively good, and it decreased with the increase of the percentage of slips. When the percentage of slips was low, the DSDM method did a wonderful job. The greater the percentage of slips became, the more advantage the SDM method would be. As a whole, the SDM method was the best. For the corrected marginal mastery ratio （MMR） and allowed one-slip corrected pattern mastery ratio （OPMR）, the values of the indexes under SDM_A method were all greater than 90%. For the corrected pattern mastery ratio （PMR）, they were greater than 75% under SDM_B method. （2） The real data study showed that there were some cognitive deficiencies on three attributes （ reference, information matching and processing level of correct response）, which was consistent with the findings of the other research. And the attribute mastery probabili- ties （AMPs） pointed out that the other attributes could still be promoted. All these showed that the G-AHM was a reliable and applica- ble model for group-level CD assessment. To conclude, the G-AHM model was an adaptive model for group assessment, and it had some advantages over the group assessment based on the individual assessment.