多维计算机化自适应测验中项目曝光控制选题策略的比较

A Comparison Study of Item Selection Strategies with Item Exposure Controlling in MCAT

在MCAT中考查四种项目选择指标在有无曝光控制条件下的选题表现。项目选择指标分别是:(1)贝叶斯的D优化方法(D-optimality)、后验期望Kullback-Leibler方法(KLP)、基于等权重复合分数的最小误差方差方法(the minimized error variance of the linear combination scorewith equal weight,V1)和基于最优权重复合分数的最小误差方差方法(the minimized error variance ofthe composite score with optimized weight,V2)。将针对认知诊断CAT项目曝光控制的的限制阈值方法(Restrictive Threshold,RT)和限制进度(Restrictive Progressive,RPG)方法、单维CAT中的最大优先指标方法(Maximum Priority Index,MPI)推广到MCAT。模拟研究表明:(1) KLP,D-优化和V1对领域分数估计准确,能力返真性比V2更好。(2)尽管V1和V2方法相比KLP和D-优化方法提高了题库利用率,但这四种选题指标都产生不均匀的项目曝光率分布。(2)三种曝光控制策略都极大地提高项目曝光均匀性,且不明显降低测量精度。(3) MPI与RPG方法在曝光控制方面表现类似,且比RT的方法表现更好。

Four item selection indexes with and without exposure control are evaluated and compared in multidimensional computerized adaptive testing(CAT).The four item selection indices are D-optimality,Posterior expectation Kullback-Leibler information(KLP),the minimized error variance of the linear combination score with equal weight(V1),and the minimized error variance of the composite score with optimized weight(V2).The maximum priority index(MPI)method for unidimensional CAT and two item exposure control methods(the restrictive threshold(RT)method and restrictive progressive(RPG)method,originally proposed for cognitive diagnostic CAT)are extended to the miltidimentional CAT.The results show that:(1)KLP,D-optimality,and V1 perform well in recovering domain scores,and all outperform V2 in psychometric precision;(2)KLP,D-optimality,V1,and V2 produce an unbalanced distribution of item exposure rates,although V1 and V2 offer improved item pool usage rates;(3)all the exposure control strategies improve the exposure uniformity greatly and with very little loss in psychometric precision;(4)RPG and MPI perform similarly in exposure control,and are both better than RT.