期刊文献+
任意字段
题名或关键词
题名
关键词
文摘
作者
第一作者
机构
刊名
分类号
参考文献
作者简介
基金资助
栏目信息

基于信赖域Newton算法的ELM网络 被引量：3

ELM based on trust region Newton method
原文传递
导出
摘要 针对极端学习机(ELM)网络伪逆输出权值计算方法的运算复杂度制约其训练速度问题,提出一种基于信赖域Newton算法的新型ELM网络(TRON-ELM),并采用信赖域Newton算法求解ELM网络的输出权值.该算法首先构造一个ELM网络代价函数的Newton方程,并将其作为一个无约束优化问题,采用共轭梯度法求解,避免了求代价函数Hessian矩阵逆的运算,提高了训练速度,信赖域条件的存在保证了算法的整体收敛性.仿真实验结果验证了所提出方法的有效性. Considering the problems that the complexity of generalized inverse limits the learning speed of extreme machine learning（ELM）,a novel ELM,called TRON-ELM,is proposed based on the trust region Newton method in which the trust region Newton method is used to derive the output weights.The proposed method takes the Newton equation of the cost funcion of ELM as an unconstrained optimization,and a conjugate gradient method is used to solve the equation,which avoids solving the inverse of the Hessian matrix,thus the operation speed is improved.Meanwhile,the existence of trust region guarantees the global convergence.The experimental results show the effectiveness of the proposed method.
作者 韩敏 王新迎
机构地区 大连理工大学电子信息与电气工程学部
出处 《控制与决策》 EI CSCD 北大核心 2011年第5期757-760,共4页 Control and Decision
基金 国家自然科学基金项目(60674073) 国家科技支撑计划项目(2006BAB14B05) 国家973计划项目(2006CB403405)
关键词 极端学习机 信赖域Newton法 共轭梯度法 回归 extreme machine learning trust region Newton method conjugate gradient method regression
分类号 TP273 [自动化与计算机技术—检测技术与自动化装置]
  • 相关文献

参考文献12

  • 1Rumelhart D E, McClelland J L. Parallel distributed processing: Explorations in the microstructure of cognition[M]. Cambridge: MIT Press, 1986.
  • 2Haykin S. Neural networks: A comprehensive foundation[M]. New Jersey: Prentice Hall, 1999.
  • 3叶军,张新华.多层前向神经网络的快速学习算法及其应用[J].控制与决策,2002,17(B11):817-819. 被引量：27
  • 4王俊年,申群太,周少武,沈洪远.基于种群小生境微粒群算法的前向神经网络设计[J].控制与决策,2005,20(9):981-985. 被引量：13
  • 5Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: Theory and applications[J]. Neurocomputing, 2006, 70(1/2/3): 489-501.
  • 6Feng G R, Huang G B, Lin Q P, et al. Error minimized extreme learning machine with growth of hidden nodes and incremental learning[J]. IEEE Trans on Neural Networks, 2009, 20(8): 1352-1357.
  • 7Huang G B, Chen L, Siew C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes[J]. IEEE Trans on Neural Networks, 2006, 17(4): 879-92.
  • 8史志伟,韩敏.ESN岭回归学习算法及混沌时间序列预测[J].控制与决策,2007,22(3):258-261. 被引量：47
  • 9Lin C J, Weng R C, Keerthi S S. Trust region Newton method for large-scale logistic regression[J]. J of Machine Learning Research, 2008, 9: 627-650.
  • 10Nash S G. A survey of truncated-Newton methods[J]. J of Computational and Applied Mathematics, 2000, 124(1/2): 45-59.

二级参考文献37

  • 1王永骥 涂健.神经元网络控制[M].北京:机械工业出版社,1999..
  • 2Brits R, Engelbrecht A P, Fan den Bergh. A Niching Particle Swarm Optimizer[A]. Proc Conf on Simulated Evolution and Learning[C]. Singapore, 2002.
  • 3Brits R, Engelbrecht A P, Fan den Bergh. Scalability of Niche PSO[A]. Proc IEEE Int Conf on Intelligence Symposium[C]. Indianapolis, 2003: 228-234.
  • 4Fredric M Ham, Ivica Kostanic. Principles of Neurocomputing for Science and Engineering[M]. McGraw Hill, 2001.
  • 5Clerc M, Kennedy J. The Particle Swarm -- Explosion, Stability, Andconvergence in a Multidimensional Complex Space[J]. IEEE Trans on Evolutionary Computation, 2002, 6(2): 58-73.
  • 6Mendes R, Kennedy J, Neves J. The Fully Informed Particle Swarm: Simpler, Maybe Better[J]. IEEE Trans on Evolutionary Computation, 2004 8(6): 204-210.
  • 7Ioan Cristian Trelea. Particle Swarm Optimization Algorithm: Convergence Analysis and Parameter Selection[J]. Information Processing Letters, 2003, 85: 317-325.
  • 8F van den Bergh. Particle Swarm Weight Initialization in Multi-layer Neural Networks[A]. Development and Pracrice of Artificial Intelligence Techniques[C]. Durban, 1999: 41-45.
  • 9F van den Bergh, Engelbrecht A P. Cooperative Learning in Neural Networks Using Particle Swarm Optimizers[J]. South Africa Computer J,2000, 26(11): 84 - 90.
  • 10F van den Bergh, Engelbrecht A P. Training Product Unit Networks Using Cooperative Particle Swarm Optimizers[A]. Proc IEEE Int Conf on Neural Networks[C]. Washington DC,2001.

共引文献84

同被引文献36

  • 1张贤达.矩阵分析与应用[M].北京:清华大学出版社,2005.341-400.
  • 2Song Q S, Feng Z R. Effects of connectivity structure of complex echo state network on its prediction performance for nonlinear time series[J]. Neurocomputing, 2010, 73(10- 12): 2177-2185.
  • 3Muhammad A E Saeed Z. Chaotic time series prediction with residual analysis method using hybrid EIman-NARX neural networks[J]. Neurocomputing, 2010, 73(13-15):2540-2553.
  • 4Huang G B, Zhu Q Y, Stew C K. Extreme learning machine: Theory and applications[J]. Neurocomputing, 2006, 70(1-3): 489-501.
  • 5Liang N Y, Huang G B, Saratchandran N, et al. A fast and accurate on-line sequential learning algorithm for feedforward networks[J]. IEEE Trans on Neural Networks, 2006, 17(6): 1411-1423.
  • 6Miche Y, Soriamaa A, Bas P, et al. OP-ELM: Optimally pruned extreme learning machine[J]. IEEE Trans on Neural Networks, 2010, 21(1): 158-162.
  • 7Liu N, Wang H. Ensemble based extreme learning machine[J]. IEEE Signal Processing Letters, 2010, 17(8): 754-757.
  • 8Lan Y, Soh C Y, Huang G B. Constructive hidden nodes selection of extreme learning machine for regression[J]. Neurocomputing, 2010, 73(16/17/18): 3191-3199.
  • 9Malathi V, Marimuthu N S, Baskar S. Intelligent approaches using support vector machine and extreme learning machine for transmission line protection[J]. Neurocomputing, 2010, 73(10-12): 2160-2167.
  • 10LeungF HF,Lam H K,Ling S H.etal.Tuning of the structure and parameters of a neural network using an improved genetic algorithm [J].IEEE Transactions on Neural Networks,2003,14(1) : 79-88.

引证文献3

二级引证文献19

控制与决策
2011年 第5期

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部