一种基于进化算法的概化理论最佳样本量估计新方法:兼与三种传统方法比较

A new method for estimating the optimal sample size in generalizability theory based on evolutionary algorithm:Comparisons with three traditional methods

概化理论在心理与教育测量领域应用较广。如何使测量程序在预算限制的情况下达到较优的可靠性是研究者需要考虑的重要问题,这个问题可以转换为最佳样本量估计的问题。提出了一种基于进化算法的估计概化理论下最佳样本量的新方法——约束进化算法,并采用模拟研究的方法比较了微分优化法、拉格朗日法、柯西不等式法等三种传统方法与约束进化算法的优劣。结果表明:在两侧面交叉设计、两侧面嵌套设计和三侧面交叉设计中都证明了约束进化算法更具优越性,建议研究者在今后的研究中优先使用。

Generalizability Theory(GT)is widely applied in psychological measurement and evaluation.A larger generalizability coefficient often indicates a higher reliability the test may have.Generalizability coefficients can be improved by increasing sample sizes.However,the size of a sample would be subject to budget constraints.Therefore,it is important to examine how to effectively determine the size of a sample considering the budget constraints.The existing literature has been largely limited to traditional methods,such as the differential optimization method,the Lagrange method and the Cauchy Schwartz inequality method.These traditional methods have limited scope of application and their typical conditions are hard to satisfy.In addition,there is no unified comparison available.Fortunately,with the increased use of high performance computing,the Constrained Optimization Evolutionary Algorithms(COEAs)becomes highly feasible.This paper expands and compares the four methods—the differential optimization method,Lagrange method,Cauchy Schwartz inequality method,and COEAs—determine the best solution to the optimal sample size problem under the budget constraints in GT.Specifically,this paper compares the applicability of the four methods using three generalizability designs,including p×i×r,(r:p)×i and p×i×r×o designs.The results are presented as follows:(1)In the optimization performance of two-facet generalizability design of p×i×r and(r:p)×i,the performance of COEAs is slightly better than that of the traditional methods,whereas the performance of three traditional methods is equivalent.Although COEAs and the traditional methods have showed similar accuracy,the former has better compliance concerning budget constraints.(2)In the optimization performance of three-facet generalizability design of p×i×r×o,the performance of COEAs is obviously better than that of the traditional methods.The least ideal generalizability coefficient is obtained using the differential optimization method,whereas its budget compliance is the best;the generalizability coefficient obtained by Lagrange method is the best,but higher than the budget.The Cauchy inequality method obtains a better generalizability coefficient under special budget constraints.But,the performance of COEAs is slightly better than that of Cauchy Schwartz inequality method,especially closer to the budget constraints.(3)In terms of the algorithm complexity,COEAs obtains an obviously smaller algorithm complexity than do the traditional methods.The complexity of the three traditional methods is relatively high.However,COEAs does not rely on the derivation of mathematical formulas,and the algorithm is relatively less complex.(4)In terms of the algorithm applicability,COEAs is significantly better than the traditional methods.The applicability of the three traditional methods is relatively narrow.However,COEAs does not rely on a specific generalizability design or a budget expression,and,therefore,the applicability of COEAs is stronger.(5)In terms of the algorithm generalizability,COEAs is obviously better than the traditional methods.The limited mathematical principles make it difficult to extend the three traditional methods to more complex generalizability designs,and thus,the feasibility of calculation is poor.Howerve,COEAs has revealed stronger generalizability.(6)In terms of the possibility of getting the best solution,COEAs is also better than the traditional methods.Because evolutionary algorithm is a probabilistic algorithm,multiple tests can be conducted to obtain better results for optimal sample sizes.Under some conditions,COEAs can determine better solutions,which,however,is impossible for three traditional methods.(7)These results suggest that COEAs is superior to three traditional methods in estimating the optimal sample size problem under the budget constraints in GT.It is recommended that researchers use COEAs in future research to determine an optimal sample size in their psychological measurement and evaluation.