两种新的多维计算机化分类测验终止规则

Two new termination rules for multidimensional computerized classification testing

计算机化分类测验(Computerized Classification Testing,CCT)由于具备分类的功能,目前在职业资格考试、健康与护理问卷等以分类为目的的测验中得到广泛应用。作为CCT的重要组成部分,终止规则不仅决定测验停止的条件而且直接影响分类准确率及测验效率。然而,目前少有研究对多维CCT(Mulitidimensional CCT,MCCT)的终止规则进行探索。针对已有MCCT终止规则的不足,提出两种新的MCCT终止规则(即基于马氏距离的多维序贯似然比规则Mahalanobis-SPRT和随机缩减的多维广义似然比规则M-SCGLR),并开展模拟研究在不同实验条件下(比如,不同的题库结构、能力维度间相关及分界函数)考查它们的表现。结果表明:(1)在使用补偿性分界函数的条件下,Mahalanobis-SPRT规则具有较高的分类精度和与同类方法相近的测验长度;(2)在几乎所有实验条件下,M-SCGLR规则不仅在测验精度上大幅优于已有的多维随机缩减规则,而且具有较短的测验长度。

Computerized classification testing(CCT) is a subset of computerized adaptive testing(CAT), and it aims to classify examinees into one of at least two possible categories that denote results such as pass/fail or non-mastery/partial mastery/mastery. Therefore, CCTs focus on increasing the accuracy of classification which is different from CATs designed for precise measurement. The termination rule is one of the key components of CCT. However, as pointed out by Nydick(2013), most CCTs(i.e., UCCTs) were designed under unidimensional item response theory(IRT), in which the unidimensionality assumption is easily violated in practice. Thus, researchers then began to construct multidimensional CCT termination rules(i.e., MCCT) based on multidimensional IRT. To date, however, these rules still have some deficiencies in terms of classification accuracy or test efficiency. Most current studies on termination rules of MCCT are based on termination rules of UCCT. In UCCTs, termination rules require setting a cut point, θ0, of the latent trait to calculate the statistics;and when they are extended from UCCT to MCCT, the cut point will become a classification bound curve or even a surface(i.e., g(θ)(28)0). At this time, a question is how to convert the curve or surface into θ0. To this end, the projected sequential probability ratio test(P-SPRT), constrained SPRT(C-SPRT;Nydick, 2013), and multidimensional generalized likelihood ratio(M-GLR) were respectively proposed to solve the problem in different ways. Among them, P-SPRT and C-SPRT choose specific points on g(θ) as the approximate cut point, θ0, by projecting into Euclidean space or constraining on g(θ) respectively;as for M-GLR, because the generalized likelihood ratio statistic can be calculated without a cut point, it can be directly employed in MCCT. To overcome the limitation that P-SPRT may lead to unstable results at the beginning of the test, this study proposed the Mahalanobis distance-based SPRT(Mahalanobis-SPRT). In addition, stochastic curtailment is a technique for shortening the test length by predicting whether the classification of participants will change as the test continues. This article also combined M-GLR with the stochastic curtailment and proposed M-GLR with stochastic curtailment(M-SCGLR). A full-scale simulation study was conducted to(1) compare both the Mahalanobis-SPRT and M-SCGLR with the P-SPRT, C-SPRT, M-GLR, and multidimensional stochastically curtailed SPRT(M-SCSPRT) under varying conditions;(2) compare the classification performance of the above six termination rules for participants with specific abilities to explore whether there is a significant difference in the sensitivity of various rules to classify specific participants. To achieve the first research objective, three levels of correlation between dimensions(ρ=0, 0.5, and 0.8), two item bank structures(within-item multidimensionality and between-item multidimensionality), and two kinds of classification boundary(compensatory boundary and non-compensatory boundary) were considered;to achieve the second objective, 36 specific ability points(θ1, θ2) were generated where θ1, θ2 ?{-0.5,-0.3,-0.1, 0.1, 0.3, 0.5}. The results showed that:(1) when the compensatory classification function was used, the Mahalanobis-SPRT led to higher classification accuracy and similar test length to the rules without stochastic curtailment;(2) under almost all conditions, the M-SCGLR not only possessed higher precision but also maintained the short test length, compared to M-SCSPRT that also uses stochastic curtailment;(3) the six termination rules showed a consistent change in the sensitivity of the precision and test length to specific participants. To sum up, two new MCCT termination rules(Mahalanobis-SPRT and M-SCGLR) are put forward in this article. Although the simulation results are very promising, several research directions merit further investigation, such as the development of MCCT termination rules for more than two categories, and the construction of MCCT termination rules by incorporating process data like the response time.