基于贝叶斯学习的惩罚因子的选择

Penalty Factors Selection Based on Bayesian Learning

文章基于贝叶斯学习,将正则化方法从贝叶斯分析的角度展开,在响应变量服从正态分布、回归系数服从指数型先验分布族的条件下,用贝叶斯准则给出了惩罚因子的取值与响应变量、系数的方差之间的关系,并将这一结果应用到岭回归和lasso回归中惩罚因子的选择。实例检验结果表明,当响应变量和系数服从正态分布,惩罚因子的值取二者方差商的方法比岭迹法和广义交叉验证法(GCV)拟合效果更优。

Based on Bayesian learning, this paper develops regularization method from the angle of Bayesian analysis. In the condition of response variable subjected to normal distribution and regression coefficients subjected to the exponential prior distri- bution family, the paper presents the relationship of penalty factors values, response variable and coefficients variance by use of Bayesian rule, and applies the result to the adoption of penalty factor of ridge regression and lasso regression. The empirical test result shows that when the response variable and coefficients follow normal distribution, as far as the fitting effect is concerned, the method for penalty factor value to select the quotient between response variable and regression coefficients variance is better than ridge trace, generalized cross validation （GCV） method.