基于仿真样本生成的极速学习机泛化能力改进算法

A learning algorithm for improving generalization capability of extreme learning machine through synthetic instance generation

极速学习机出色的训练速度和泛化能力受到了广泛的关注,已有的针对于提升极速学习机泛化性能的学习算法主要集中于优化其框架结构,增加了模型的复杂度并容易产生过拟合.提出一种基于仿真样本生成策略的极速学习机泛化能力改进学习算法(Extreme Learning Machine Generalization Improvement through Synthetic Instance Generation,SIGELM),该算法不需要修改极速学习机的框架结构(包括输入层权重、隐含层偏置、隐含层节点个数、隐含层节点激活函数类型等),而是利用与训练集中高不确定性训练样本近似同分布的仿真样本优化极速学习机的输出层权重.为了获得符合要求的仿真样本,SIGELM在高不确定性训练样本的邻域内选择能够增加极速学习机训练表现的仿真样本.实验结果证实该算法显著地改进了极速学习机的泛化能力,同时有效地控制了极速学习机的过拟合.

Extreme Learning Machine（ELM）is known to be a promising algorithm which makes learning speed extremely fast and thus attracts a lot of attentions from both academia and industry.However,due to the random determination of input-layer weights and biases for hidden nodes,it might generate some un-optimal parameters which may have a negative influence on the generalization performance and predicted robustness of ELM.To alleviate such weakness,a number of works have been proposed to further improve the generalization capability and stability of ELM.These existing methods principally focus on modifying or enhancing the structure of ELM and tend to suffer from over-fitting problem for paying excessive attention on matching the entire training dataset as well as increasing the model’s complexity substantially.In this paper,we proposed an effective learning algorithm（Extreme Learning Machine Generalization Improvement through Synthetic Instance Generation,SIGELM）to improve the generalization performance of ELM based on randomly generated synthetic instances.SIGELM does not need to modify the architecture of ELM model and uses the synthetic instances of which the distribution is approximately equal to the training instances with high uncertainty to optimize the output-layer weights of ELM.In order to obtain the required synthetic instances,a neighborhood is determined for each high-uncertainty training sample and then the synthetic instances which enhance the training performance of the current updated ELM on the initial training dataset are selected in the neighborhood.The experimental results on four representative KEEL regression datasets demonstrated that SIGELM can significantly improve the generalization performance of ELM and meanwhile effectively control its over-fitting,which verified the feasibility and effectiveness of our proposed SIGELM algorithm.