基于GPCM的计算机自适应测验选题策略比较

Item Selection Strategies for Computerized Adaptive Testing with the Generalized Partial Credit Model

选题策略是计算机自适应测验(Computerized Adaptive Testing,CAT)研究的一项重要内容,它的好坏直接关系到考试的信度、效度及考试的安全性。CAT的许多研究与应用,都建立在0-1二级评分模型基础上,对多级评分CAT的选题策略的研究很少报导。目前国内虽已开展了基于GRM的CAT研究,但基于GPCM的CAT的研究尚未见有关报道。本文通过计算机模拟程序,对基于拓广分部评分模型(Generalized Partial Credit Model,GPCM)下的CAT的四种选题策略在多种情况下进行了比较研究。研究结果表明:被试能力呈正态分布时,选题策略的使用效果与项目步骤参数分布有很大的关系。(1)项目步骤参数均服从正态分布时,采用能力与项目步骤参数匹配选题策略效果最佳;(2)项目步骤参数均服从均匀分布时,能力与项目步骤参数平均数匹配选题策略效果最佳。

The objective of computerized adaptive testing （CAT） is to construct an optimal test for each examinee. Item Selection Strategy （ISS） is an important part of CAT research, whose quality is directly related to the reliability, efficiency, and security of the test. Many researches and applications of CAT are based on a dichotomously scored model. It is highly evident that more information can be obtained from examinees using a polytomously scored model rather than a dichotomous model. Moreover, it is necessary for us to further explore CAT research based on a polytomously scored model. Both the Generalized of a polytomously scored Partial Credit Modal （GPCM） and the Graded Response Model （GRM） are within the range model. However, they differ from each other. In the GRM, the item grade difficulties ascend monotonously as the grades increase; while the GPCM shows the performing process of the item, which is separated into some line-steps to put forwards. In the GPCM, each item contains several step parameters, and there are no specific rules governing them. The posterior step cannot advance when the earlier step has not been completed, and the posterior＇s step parameter may be lower than that of the previous one. Considerable research is already being conducted on CAT using the GRM; however, in our country, there are few reports pertaining to research on CAT using the GPCM. This study investigated the four types of ISS in comparison with CAT in various circumstances, using the GPCM through computer simulated programs. They are implemented in four item pools, and each item pool has a capacity of 1000 items. Each item has five step parameters; further, the discrimination parameter and step parameters are distributed as follows： b - N （0,1）, lna - N （0,1）, b - N （0,1）, a - U （0.2,2.5）, b - U （-3,3）, lna - N （0,1）, b - U （-3,3）, and a - U （0.2,2.5）. Item parameters are generated based on the Monte Carlo simulation method. Responses to the items are generated according to the GPCM for a sample of 3000 simulatees θ - N（0,1） whose trait level was also generated using the Monte Carlo simulation method in some types of ISS. During the course of responses, the simulatees＇ ability is estimated based on the response obtained. In addition, after the four item pools are sorted by the discrimination parameter to complete the a -stratified design, the abovementioned process is performed repeatedly. Thirty-two simulated CATs are administered with the output evaluated with regard to the following measurements： precision, ISS steady, item used even, average use of item per person, χ^2, efficiency, and item overlap. The data in tables 1 and 2 include both the index values used for evaluation （which were obtained from the CAT process using four types of ISS when the item pool did not adopt the stratified design and instead adopted the a-stratified design） and values that are calculated after summing the weight of every index value： We can draw the following conclusions from the data in the tables： all the ability estimates are highly accurate and have fewer differences. Moreover, we compare the value by summing every means weight, we learn that the item step parameter distribution greatly influences the choices of ISS. On the condition that the examinee＇s trait level follows normal distribution, the application results of the ISS and the item step parameter distribution share a very close relationship. （1） If the item＇s step parameters follow a normal distribution, the efficiency of the ISS for a random step parameter matching the trait level is much better than that for others. （2） If the item＇s step parameters follow a uniform distribution, the efficiency of the item selection strategy for the item＇ s average step parameter matching the trait level is much better than that for others.