一种混合计分的非参数认知诊断方法:曼哈顿距离判别法

Approach to Cognitive Diagnosis:The Manhattan Distance Discriminating Method

研究提出了一种简洁的适于混合计分的非参数认知诊断方法—曼哈顿距离判别法(MDD),将HDD纳入MDD框架,通过模拟和实证研究考察了MDD的适宜性,结果表明:(1)MDD简单易解,适于混合计分情境,0-1计分时,HDD是MDD的一个特例;(2)MDD的判准率较高,三种判别方法 R_MDD、B_MDD和W_MDD差异极小;(3)MDD具有非参数方法不受知识状态影响、对样本容量无依赖,与属性个数关系不大等特点;(4)MDD在实践中的应用效果较好,为CDA走向实践、走向课堂提供了可能。

Existing methods for fitting cognitive diagnosis models to educational test data and assigning examinees to proficiency classes are based on parametric estimation methods such as expectation maximization(EM)and Markov chain Monte Carlo(MCMC)that frequently encounter difficulties in practical applications.In response to these difficulties,non-parametric classification techniques have been proposed as heuristic alternatives to parametric procedures.Among non-parametric classification techniques,the distance discriminating method is especially simple and easy to understand.It classifies examinees by minimizing a distance measurement between observed responses(i.e.,examinee’s test item scores),and the ideal responses for a given attribute profile that would be implied by the item-by-attribute association matrix.However,studies on the non-parametric classification techniques are recently developed and there are just a few studies about the distance discriminating method.Most of these studies set a 0-1 scored situation.However,in the wake of educational test reform and the diversity of test forms,the scores of test items are not only 0 or 1.And the methods which are developed in that kind of scoring are not suitable and will lose much information of examinee’s responses,which can lead to lower classification accuracy.Although a generalized distance discriminating method(GDD)has been extended to test with polytomous responses,the computation of the generalized distance is weighted by the examinee’s item response probability of item response theory which makes the method complicated and difficult to understand.An extension of clustering algorithm to polytomous diagnostic situation has made diagnosis assessment easier to generalize,but a calculation of attribute-score is not easy to operate and the class of clustering may be unclear.Taking advantage of a simpler and more general distance-Manhattan Distance which can be used to measure the difference between the observed response pattern and the ideal response patterns in cognitive diagnosis,this study proposes a general distance based discriminating method which is easier to operate,understand and clarify.Both Monte Carlo simulations and a real data analysis are used to investigate the effectiveness and practicability of the MDD.Results demonstrate that:(1)The existing hamming distance discriminating method(HDD)is a special case of the proposed Manhattan distance discriminating method(MDD)which can be used for test with both binary and polytomous scoring settings or just one of them.(2)MDD has high classification accuracy,and three discriminating methods of MDD have almost the same rate of correct classification and can be chosen according to the actual demand.(3)Consistent with the existing nonparametric method,the sample size and distributions of knowledge state have little effect on the MDD classification accuracy;As the number of attributes increases,the classification accuracy drops slowly.(4)MDD has good empirical validity,and the classification result of MDD has a high degree of agreement with the actual situation.