基于PAC-Bayes边界理论的SVM模型选择方法

Method of SVM model selection based on PAC-Bayes bound theory

PAC-Bayes边界理论融合了贝叶斯定理和随机分类器的结构风险最小化原理,它作为一个理论框架,能有效评价机器学习算法的泛化性能。针对支持向量机(SVM)模型选择问题,通过分析PAC-Bayes边界理论框架及其在SVM上的应用,将PAC-Bayes边界理论与基于交叉验证的网格搜索法相结合,提出一种基于PAC-Bayes边界的SVM模型选择方法(PBB-GS),实现快速优选SVM的惩罚系数和核函数参数。UCI数据集的实验结果表明该方法优选出的参数能使SVM具有较高的泛化性能,并具有简便快速、参数选择准确的优点,能有效改善SVM模型选择问题。

PAC-Bayes risk bound integrating theories of Bayesian paradigm and structure risk minimization for stochastic classifiers has been considered as a framework for effective evaluating the generalization capability of machine learning algorithms. Aiming at the problem of model selection of SVM, this paper analyzes the theoretical framework of PAC-Bayes bound and its application to SVM, and combines the PAC-Bayes bound with grid search method based on cross validation.A method of model selection based on PAC-Bayes bound（PBB-GS）is put forward to select the best penalty parameter and kernel parameter rapidly. From the experimental results of the UCI datasets, it draws the conclusion that the parameters selected by PBB-GS can make SVM achieve better generalization performance, and this method is simple, fast and accurate, which can improve the model selection of SVM effectively.