认知诊断中基于条件期望的距离判别方法

Distance Discrimination Method based on Conditional Expectation in Cognitive Diagnosis

分类是认知诊断评估的一个核心问题。基于观察反应模式与理想反应模式之间的距离的判别方法,以确定性的理想反应模式为类中心,而这没有考虑误差,故未充分利用总体分布信息。为了更充分地利用总体分布信息、提高诊断分类效果和拓展诊断评估的适用性,本研究提出给定知识状态条件下项目反应模式的条件期望向量为类中心的欧氏距离判别方法,同时提出认知诊断模型下项目反应函数估计方法以获得这个条件期望向量。模拟研究表明:认知诊断模型下的项目反应函数估计方法得到的条件期望向量返真性较高,获得的分布信息较准确;在观察反应模式与理想反应模式差异大的情形下,基于条件期望向量为类中心的欧氏距离判别方法优于基于理想反应模式为类中心的分类方法(广义距离方法和非参数方法)。研究可为认知诊断分类和等值方法提供一个参考。

The primary purpose for cognitive diagnostic assessment is to classify examinees into mutually exclusive categories. The current practice of obtaining classification categories relies on the distance between the ideal and observed response patterns, such as generalized distance discrimination method （Sun et al., 2011, 2013） and nonparametric approach （Chiu and Douglas, 2013）. In these methods, an appropriate set of ideal response patterns can be computed from the universal set of knowledge states and a Q matrix （Tatsuoka, 1995; Leighton et al., 2004; Ding et al., 2009, 2010）. However, the ideal response pattern is generated for each knowledge state without considering the stochastic nature of item response, which is contrary to real test situations. For example, examinees who have mastered some of attributes required to solve a particular item can have a higher probability of answering it correctly than less able examinees having mastered none of the attributes. The purpose of this study is focused on choosing the center for each of the classification categories or clusters that is representative of the data. Since this task probably requires knowledge on the distribution of the data, as a practical matter, it is often reasonable to assume that item response pattern is a random vector with a discrete conditional distribution for each knowledge state. The conditional expected vector is considered to be class-center. Once the center is decided for each knowledge state, observed responses pattern can be assigned to the closest class according to minimum Euclidean distance classifier, well-known for measuring the distance between observed response pattern and the center. This method is called distance discrimination method. The conditional distribution can be defined by item response function （IRF） in cognitive diagnostic models. However, many existing item banks are developed under the framework of item response theory. In this case, we propose a method utilizing the information of the item characteristic curve （ICC） or IRF of item response theory model to estimate the IRF for cognitive diagnostic model. It is based on a nonparametric regression approach to transform the IRF of item response theory model into that of cognitive diagnostic model. The resulting IRF is used to classify examinees using minimum Euclidean distance classifier. To investigate whether this method can work under certain conditions, simulated data were generated with six attributes. Four important factors were considered： （a） the source of the attribute structure （the linear hierarchy, the convergent hierarchy, the divergent hierarchy, the unstructured hierarchy, the independent hierarchy）, （b） the number of examinees （N = 300, 500, 1,000）, （c） two cognitive diagnostic models （the deterministic inputs, noisy ＂and＂ gate model and the reduced reparametrized unified model）;（d） the quality of the items （s, g - U （0.05, 0.25） or U （0.05, 0.4）, π^＊-U （0.8, 0.98） and r^＊-U （0.1, 0.6） or ;r^＊-U （0.75, 0.95） and r^＊-U （0.2, 0.95））. The results show that the estimation method of IRF is promising in terms of precision, and distance discrimination method based on conditional expectation works well in terms of accuracy, especially when the R-RUM fitted the data with low quality of test items. In addition, the fact that the IRF transform method between cognitive diagnostic model and item response model may contribute immensely to test equating of cognitive diagnostic tests. In discussion, we also explain the relationship between nonparametric approach, generalized distance discrimination method and rule space method.