深度模型的持续学习综述:理论、方法和应用

A Survey of Continual Learning with Deep Networks:Theory,Method and Application

自然界中的生物需要在其一生中不断地学习并适应环境,这种持续学习的能力是生物学习系统的基础。尽管深度学习方法在计算机视觉和自然语言处理领域取得了重要进展,但它们在连续学习任务时面临严重的灾难性遗忘问题,即模型在学习新知识时会遗忘旧知识,这在很大程度上限制了深度学习方法的应用。持续学习研究对人工智能系统的改进和应用具有重要意义。该文对深度模型的持续学习进行了全面回顾。首先介绍了持续学习的定义和典型设定,阐述了问题的关键。其次,将现有持续学习方法划分为基于正则化、基于回放、基于梯度和基于网络结构4类,分析了各类方法的优点和局限性。同时,该文强调并总结了持续学习领域的理论分析进展,建立了理论与方法之间的联系。此外,提供了常用的数据集和评价指标,以公正评判不同方法。最后,从多个领域的应用价值出发,讨论了深度持续方法面临的问题、挑战和未来研究方向。

Biological organisms in nature are required to continuously learn from and adapt to the environment throughout their lifetime.This ongoing learning capacity serves as the fundamental basis for the biological learning systems.Despite the significant advancements in deep learning methods for computer vision and natural language processing,these models often encounter a serious issue,known as catastrophic forgetting,when learning tasks sequentially.This refers to the model’s tendency to discard previously acquired knowledge when acquiring new information,which greatly hampers the practical application of deep learning models.Thus,the exploration of continual learning is paramount for enhancing and implementing artificial intelligence systems.This paper provides a comprehensive survey of continual learning with deep models.Firstly,the definition and typical settings of continual learning are introduced,followed by the key aspects of the problem.Secondly,existing methods are categorized into four main groups:regularization-based,replay-based,gradient-based and structure-based approaches,with an outline of the strengths and weaknesses of each group.Meanwhile,the paper highlights and summarizes the theoretical progress in continual learning,establishing a crucial nexus between theory and methodology.Additionally,commonly used datasets and evaluation metrics are provided to facilitate fair comparisons among these methods.Finally,the paper addresses current issues,challenges and outlines future research directions in deep continual learning,taking into account its potential applications across diverse fields.