基于贝叶斯优化和组合模型的XGBoost改进方法

XGBoost Improved Method Based on Bayesian Optimization and Combination Model

回归预测中,模型和超参数对预测结果有着极大影响。合适的模型能够决定预测的效果,同时模型的超参数直接控制了训练的方式,采用合适的超参数能够进一步提升算法的精确度。为了提升回归预测精度和鲁棒性,提出了一种基于贝叶斯优化和组合模型的回归优化方法。首先利用贝叶斯优化快速的全局搜索能力,以交叉验证平均得分为目标函数值,对XGBoost算法和LightGBM算法进行调优,选择较好的超参数值建立BO_XGBoost和BO_LightGBM模型,接着利用序列最小规划算法确定BO_XGBoost和BO_LightGBM的模型权重并进行模型组合。在UCI公共数据集上进行的实验结果表明,该方法能够有效提高预测精度和鲁棒性。

In the field of regression prediction,models and hyperparameters have a great influence on the prediction results.The appropriate model can determine the prediction effect,while the hyperparameters of the model directly control the way of train⁃ing,and the use of appropriate hyperparameters can further improve the accuracy of the algorithm.In order to further improve the prediction accuracy and robustness of the model,a regression optimization method based on Bayesian optimization and combination model is proposed.Firstly,the fast global search ability of Bayesian optimization is used to optimize XGBoost algorithm and LightG⁃BM algorithm with the average score of cross validation as the objective function value,a better super parameter value is chosen to establish BO_XGBoost and BO_LightGBM model,and then the sequential minimum programming algorithm is used to determine the BO_XGBoost and Bo_LightGBM model weight for model combination.On UCI data sets and the results show that the method can improve the prediction accuracy and robustness effectively.