用层级相合性指标探测反应数据中噪音大小

Using Hierarchical Consistency Indexes to Evaluate the Size of the Noise in the 0-1 Response Data of Cognitive Diagnostic Test

不同的认知诊断模型(CDM)对反应数据中噪音的抗干扰能力不同,在评估CDM性能的模拟实验中,反应数据中所含噪音的大小是十分重要的实验条件。本文在认知模型已知条件下,欲使用MHCI和NHCI指标评估认知诊断测验的反应数据(0, 1评分)中噪音的大小。模拟实验表明,两指标与噪音存在明显的统计规律。尤其是以NHCI为主要自变量对噪音进行预测的回归方程中,回归模型解释率均接近90%;以此实现对噪音的有效预测,从而为选择CDM提供一个参考。

Assessing the performance of cognitive diagnosis model(CDM) in the simulation experiments, the size of the noise in the response data is a very important experimental condition. Due to the hidden noise, it becomes difficult to choose the corresponding CDM in applications. In this article, the modified hierarchical consistency index(MHCI) and new hierarchical consistency index(NHCI) are used to evaluate the size of the noise in the 0-1 response data of cognitive diagnostic test. In order to predict the size of the noise in the response data, Monte Carlo simulation experiment is carried out to find quantitative regularity between the indexes(MHCI, NHCI) and the noise.Provide that the reduced Q matrix(Qr) is with K-row and M-column, and the test Q matrix Qt is pile of L-matrix Qr, that is Qt=(Qr,...,Qr), where L=1, 2, 3, 4, respectively. L influences the test length essentially. The experiment investigates the changes of the mean values of MHCI(MVM) or NHCI(MVN)) regulated with different attribute structures(linear, convergent, divergent, unstructured model) under the condition of different numbers of attributes(5, 6, 7) × different numbers of Qr(L=1, 2, 3, 4) × different sizes of the noise(slippage belongs in {.30.25.20.15.10.05}). For unstructured model, simulation experiment is carried out with only 5 attributes. For the other K=6, 7, the amount of computing is too heavy to implement. This means the number of attributes in unstructured model is constant.First, in order to get quantitative regularity between the indexes(MHCI, NHCI)and the noise, through stepwise regression to build the regression equations with MVM or MVN as the dependent variable, slippage ratio, the number of Qr and the number of attributes as the independent variable. Experimental results show that the slip ratio and the number of Qr significantly affect the MVM or MVN. Slippage ratio is the main influence factor;The greater the slippage ratio or the larger times of tests, the smaller the MVM or MVN will be. Number of attributes in most cases can enter the regression equations, but values of the effect are generally small. Regardless of the kind of attribute structures, the two factor regression models have good explanation rate, especially for NHCI index whose rate is above 90%. There is a stable and significant quantitative regularity between slip ratio and index, which provides a way to predict slippage ratio.In order to predict the size of the noise hidden in the data, inverse regression with slippage ratio is used as the dependent variable, one of MVM or MVN, the number of Qr and the number of attributes are used as the independent variable, the regression equations are built through stepwise regression. The experimental results show that the linear model is similar to the convergent model, MVM or MVN, the number of Qr in the regression models have bigger effect. In these two models, the explanation rate of the first factor variance is more than 65%, the explanation rate of the two factors is close to 89%. The results for divergence and unstructured are similar: MVN and the number of Qr in the regression models have bigger effect. In these two models, the explanation rate of the first factor variance is close to 83%, the explanation rate of the two factors is more than 92%. In conclusion, in experimental conditions of similar cases, just using the above two factor regression models to estimate the slippage ratio can achieve good effect.