类别增量学习研究进展和性能评价

Class Incremental Learning:A Review and Performance Evaluation

机器学习技术成功地应用于计算机视觉、自然语言处理和语音识别等众多领域.然而,现有的大多数机器学习模型在部署后类别和参数是固定的,只能泛化到训练集中出现的类别,无法增量式地学习新类别.在实际应用中,新的类别或任务会源源不断地出现,这要求模型能够像人类一样在较好地保持已有类别知识的基础上持续地学习新类别知识.近年来新兴的类别增量学习研究方向,旨在使得模型能够在开放、动态的环境中持续学习新类别的同时保持对旧类别的判别能力(防止“灾难性遗忘”).本文对类别增量学习(Class-incremental learning,CIL)方法进行了详细综述.根据克服遗忘的技术思路,将现有方法分为基于参数正则化、基于知识蒸馏、基于数据回放、基于特征回放和基于网络结构的五类方法,对每类方法的优缺点进行了总结.此外,本文在常用数据集上对代表性方法进行了实验评估,并通过实验结果对现有算法的性能进行了比较分析.最后,对类别增量学习的研究趋势进行展望.

Machine learning has been successfully applied in many fields such as computer vision,natural language processing,and speech recognition.However,in the current machine learning systems,models are often fixed after training.Consequently,they can only generalize to classes that appear in the training set,and cannot learn newly emerged classes continuously.In real-world applications,new classes or tasks will appear continuously,which requires the model to continuously learn new knowledge without forgetting the knowledge of previous seen classes.The emerging research direction of class incremental learning aims to enable models to continuously learn new classes while preserving the discrimination ability of old classes(defying“catastrophic forgetting”)in the open and dynamic environment.This paper provides a comprehensive overview of class incremental learning(CIL)developed in recent years.Specifically,existing methods are grouped into five categories:parameter regularization based,knowledge distillation based,data replay based,feature replay based and network structure based methods.The advantages and disadvantages of each method are summarized.In addition,extensive experiments are conducted to evaluate and compare those representative methods on benchmark datasets.Finally,this paper prospects the future research directions of class incremental learning.