光滑孪生参数化不敏感支持向量回归机

Smooth Twin Parametric Insensitive Support Vector Regression

作为机器学习方法之一的孪生参数化不敏感支持向量回归机(TPISVR)有着简洁的数学模型,良好的学习性能,特别适合于求解带有结构异方差噪声的数据回归问题,然而TPISVR的训练速度较低,训练效率有待提高。TPISVR的传统算法可以归结为通过转化对偶问题的方法求解2个带有不等式约束的二次规划问题,然而这种求解二次规划问题的方法对于样本数目较大的问题将受到时间和内存的制约,这是导致TPISVR训练效率低的关键所在。针对此问题,首先,引入正号函数,将TPISVR的2个二次规划问题转化为2个不可微的无约束优化问题;其次,引入CHKS光滑函数和正则项,对TPISVR模型进行正则化,并对不可微的无约束优化问题进行光滑逼近,从而将不可微的模型转化为可微的无约束优化问题,并用收敛速度快的Newton-Armijo方法求解新模型,提出光滑孪生参数化不敏感支持向量回归机(STPISVR);最后,从理论上证明了STPISVR模型是收敛的,并具有任意阶光滑性。为了验证所提算法的有效性和可行性,对机器学习常用的人工数据集和UCI数据集进行仿真实验。实验结果表明:和其他机器学习方法相比,STPISVR在保证精度不下降的前提下,获得了更高的训练效率。

As one of the machine learning methods,twin parametric insensitive support vector regression(TPISVR)had a simple mathematical model and good learning performance.It was especially suitable for solving data regression problems with structural heteroscedasticity noise.However,the training speed of TPISVR was low,and the training efficiency needs to be improved.The traditional algorithm of TPISVR could be reduced to solve two quadratic programming problems with inequality constraints by transforming dual problems.However,this method of solving quadratic programming problems with large number of samples would be restricted by time and memory,which was the key to the low training efficiency of TPISVR.In this study,the positive sign function was introduced to transform the two quadratic programming problems of TPISVR into two non-differentiable unconstrained optimization problems.Secondly,CHKS smooth function and regular term were introduced to regularize TPISVR model,and smooth approximation was made to the non-differentiable unconstrained optimization problem,so as to transform the non-differentiable model into a differentiable unconstrained optimization problem.The new model was solved by Newton Armijo method with fast convergence speed,and a smooth twin parameterized insensitive support vector regression machine(STPISVR)was proposed.Finally,it was proved theoretically that STPISVR model was convergent and had arbitrary order smoothness;In order to verify the effectiveness and feasibility of the algorithm,simulation experiments were carried out on the artificial data set and UCI data set commonly used in machine learning.The experimental results showed that compared with other machine learning methods,STPISVR achieved higher training efficiency without reducing the accuracy.