结合题目作答时间的计算机化自适应测验选题方法

The Use of Response Time in Item Selection of Computerized Adaptive Testing

计算机形式的测验能够记录考生在测验中的题目作答时间(RT),作为一种重要的辅助信息来源,RT对于测验开发和管理具有重要的价值,特别是在计算机化自适应测验(CAT)领域。本文简要介绍了RT在CAT选题方面应用并作以简评,分析了这些技术在实践中的可行性。最后,探讨了当前RT应用于CAT选题存在的问题以及可以进一步开展的研究方向。

The computer-based test enables the examinee’s response time(RT)to be recorded accurately.As an important source of auxiliary information,RT have an important potential value for test development and management,especially in the field of Computerized Adaptive Testing(CAT).With the collection of RT,the CAT assessment process can be further improved in terms of precision,fairness,and minimizing costs.It is widely known that item selection is the key step of CAT,which reflects its"adaptive"characteristics.The traditional CAT item selection algorithm does not consider RT information,this is unfavorable for test management and may lead to biased assessment results.This paper synthetically and briefly introduces the application of RT in the item selection of CAT and analyzes the feasibility of these techniques in practice,which makes the readers have a specific and clear understanding of the potential value of RT in CAT.Since item selection in CAT is based on the candidate's ability estimation(except for the selection of initial items),the improvement of ability estimation can also be considered as an indirect improvement of the item selection.Therefore,this paper divides relevant methods into two categories:(1)indirect improvement of item selection by RTs(ability estimation)and(2)direct improvement of item selection by RT(item selection method).Generally,a majority of tests are a mixture of speed and power components,while the RT provide information not only examinees’ability but also item characteristics.In the past decades,a lot of models for RT and response accuracy(RA)has been proposed(e.g.,Thissen,1983;Wang&Hanson,2005;van der Linden,2007),which makes it possible to use RT to improve the accuracy of ability estimation in CAT,and the item selection is further improved(van der Linden,2008).In general,examinees with the same ability level may need different time to complete an item(van der Linden et al.,1999),and the response time of an examinee for different items may also be different because some items are usually more time consuming than others(Veldkamp,2016).Test speededness results in examinees taking different amounts of time to complete a test.However,most standardized tests often set a specific time for practical administration purposes,when candidates are pressured by the time limit,they may improve the response speed at the expense of accuracy(Entink et al.,2009),which leads to biased ability estimation.Therefore,it is necessary to eliminate the influence of the speed factor for the test whose main goal is to evaluate ability,and this is more in line with the unidimensional hypothesis of IRT.However,the conventional item selection methods didn’t take this into account,and RT information should be introduced into the process of item selection to address this problem(van der Linden et al.,1999;Fan et al.,2012).With the development of measurement theory and technology,researchers hope to get richer diagnostic information about an examinee from the test,rather than simply evaluating him on an abstract scale,and the application of RTs is a promissing approach.