脑启发式持续学习方法:技术、应用与发展

Brain-inspired Continuous Learning:Technology,Application and Future

深度学习模型面对非独立同分布数据流时,新知识会覆盖旧知识,造成其性能大幅度下降。而持续学习(CL)技术可以从非独立同分布数据流中获取增量可用知识,不断积累新知识的同时无须从头学习,通过模仿类脑的学习与记忆机制达到类人智能。该文针对脑启发式持续学习方法进行综述。首先,回顾持续学习发展历程;其次,从类脑持续学习机制的角度,将持续学习研究方法分为经典方法与脑启发方法两类,对重放、正则化与稀疏化3种经典持续学习方法的研究现状进行总结,分析了其所面临的困境。为此,针对更接近类脑持续学习能力的突触、双系统、睡眠及模块化4类脑启发方法进行阐述分析与对比总结;最后,概述脑启发式持续学习的应用现状,并探讨了在现有技术条件下实现脑启发式持续学习所面临的挑战及其未来发展方向。

Deep learning model facing the non-independent and identically distributed data streams,the old knowledge will be covered by new knowledge,resulting in a significant performance degradation of model.Continuous Learning(CL)can acquire incremental available knowledge from non-independent and identically distributed data streams,continuously accumulate new knowledge without learning from scratch,and achieve human intelligence by imitating brain learning and memory mechanisms.In this paper,the brain-inspired continuous learning methods are reviewed.Firstly,the history of continuous learning is reviewed.Secondly,from the perspective of brain continuous learning mechanism,the research methods of continuous learning are divided into general methods and brain-inspired methods.The current research status of replay,regularization and sparsity,which are commonly used as the methods of continuous learning,are summarized,and their difficulties are analyzed under the existing technical conditions.To this end,four types of brain-inspired methods:synaptic,dual system,sleep and modularization,which are closer to the ability of brain continuous learning,are meticulously analyzed and compared.Finally,the application status of brain-inspire continuous learning are summarized,and the challenges and development of brain-inspire continuous learning under the existing technical conditions are discussed.