黑箱优化导向的序贯均匀设计

EI-SNTO Method for Black-box Optimization

在试验设计中,序贯均匀设计(sequential uniform design)是一种基于区域压缩思想的序贯空间填充设计,又可称为序贯数论优化方法(简记为SNTO),常被实际工作者用来寻求黑箱优化问题的全局最优值。该算法的核心思想是在每阶段的压缩子区域内迭代散布低偏差序列,如数论格子点(number-theoretic net),均匀设计(uniform design)等。原始的SNTO算法存在两个缺点:1)它是一种纯粹的区域压缩搜索算法,搜索过程中未使用任何统计代理模型信息,只关注试验点本身的信息;2)迭代过程中,非最优试验点的信息被完全丢弃,未被充分利用。本文引入序贯自适应试验设计思想,帮助SNTO算法更好的确定区域压缩中心,并称改进后的算法为EI-SNTO方法。该算法通过建立高斯过程代理模型,采用期望提高准则(expected Improvement,EI)和重要性抽样来帮助选择和更新试验点。一些经典的优化检验函数模拟结果验证了EI-SNTO算法的优化性能,同时本文还展示了该算法在机器学习模型(包括支持向量机和人工神经网络)超参数优化中的优良表现。

Sequential uniform design,or named as sequential number-theoretic method for optimization(SNTO),is a sequentially zoom-in space-filing design for black-box optimization.The main thrust of this method is to scatter experimental points evenly at the subsequent sub-spaces for the sake of getting the global optimum.SNTO method only focuses on the best experimental point,ignoring the information on other experimental points.It makes this algorithm being trapped to a local optimum easily.In this paper,integrating the scheme of SNTO and sequential adaptive design,another design method,called“EI-SNTO”,is proposed.The strategies of expected improvement(El) and importance sampling are employed to augment the new runs per cycle when the Gaussian Process surrogate model is embedded into the search process.In addition,numerical studies are conducted to validate the efficiency of our EI-SNTO method for global optimization.The outstanding performance of the proposed algorithm in hyper-parameter optimization for machine learning model,including support vector machine(SVM) and neural networks,is also elaborated.