基于LHD与GPR的机器学习超参数建模及优化

Machine Learning Hyperparameter Modeling and Optimization Based on LHD and GPR

针对机器学习超参数与算法性能之间的非线性、高阶交互作用和约束的复杂作用关系,文章提出一种基于高斯过程回归和遗传算法相结合的建模及优化方法。采用超拉丁方抽样在超参数配置空间内进行取点设计,建立超参数与算法性能之间的高斯过程回归模型并对所构建的模型进行遗传算法寻优。仿真研究表明,该方法能够在超参数配置空间内较好地拟合超参数与算法性能之间的复杂作用关系,与传统响应曲面法相比,在超参数配置空间内以较少的实验次数实现了全局最优,从而有效提升了优化的效率和精度。

For the nonlinear,high-order interaction and complex constraints between machine learning hyperparameters and algorithm performance,this paper proposes a modeling and optimization approach based on Gaussian process regression and genetic algorithm.The Latin hypercube sampling is used to take points in the hyperparameter feasible region,the Gaussian process regression model between the hyperparameter and the algorithm performance is established,and the genetic algorithm is utilized to optimize the established model.The simulation study shows that this method can better fit the complex relationship between hyperparameters and algorithm performance in the hyperparameter feasible region.Compared with the traditional response surface method,the proposed method can achieve global optimization with fewer experiments in the hyperparameter feasible region,thereby effectively improving the efficiency and accuracy of optimization.