LS-Ensem:一种用于回归的集成算法

LS-Ensem:A Ensemble Method for Regression

借鉴Friedman提出的基于函数空间的梯度下降搜索的思想,提出了一种新的集成学习算法———LSEnsem算法.该算法只要求个体函数满足一个很宽松的条件,从而避免了每轮迭代中寻找最优个体函数所需的大量计算,显著地降低了算法的计算复杂性.理论分析表明该算法具有指数级收敛速度以及良好的泛化性能,文中还给出了泛化误差的界.仿真结果验证了理论分析的结论,并且还显示出LSEnsem算法能够有效地抑制过拟合发生.

An novel ensemble learning method for regression, named the LS Ensem algorithm, is presented in this paper by extending the idea of gradient descent search in base function spaces proposed by Friedman. In order to avoid the large amount of calculation needed for finding an optimal base function in each iteration of Friedman＇s algorithm, base regression functions are only needed to satisfy a very loose condition in the LS-Ensem algorithm, greatly reducing the computation complexity of the algorithm. Theoretical analysis shows that the LS-Ensem algorithm con- verges exponentially, and yields an integrated regression function with good generalization property. The relationship between the number of iterations and the generalization error of the integrated regression function is also derived in this paper. A simulation validates the theoretical results, and shows that the LS-Ensem algorithm can avoid overfitting effectively.