摘要
为提高现有增量学习模型在容量固定环境下学习的持久性,提出一种基于权值选择策略的增量学习方法。根据贝叶斯神经网络携带的不确定性动态地调整权值的学习率,以此优化一个能同时记忆新旧知识的模型;为使模型的学习与记忆更有弹性,在此基础上提出一种权值选择策略,该策略可以令模型主动选择性地释放部分网络资源,在不严重损害旧任务性能的前提下促进后续任务的学习。实验结果表明,在模型容量固定的环境下,权值选择策略的引入可以更有效地发掘模型的持续学习能力。
To improve the learning persistence of the existing incremental learning models in the capacity fixed environment,a Bayesian incremental learning method based on weight selection strategy was proposed.The learning rate was dynamically adjusted according to the uncertainty carried by Bayesian neural networks,and a model that could remember the old and new know-ledge simultaneously was optimized.To explore the continuous learning ability of the model more fully,a weight selection strategy was proposed,which guided the model to selectively release some weights to promote the learning of future tasks without serious-ly damaging the performance of the old tasks.Experimental results show that the resource releasing mechanism can more effectively explore the continuous learning ability of the model under the fixed capacity environment.
作者
莫建文
朱彦桥
欧阳宁
林乐平
MO Jian-wen;ZHU Yan-qiao;OU Yang-ning;LIN Le-ping(Key Laboratory of Cognitive Radio and Information Processing of Ministry of Education,School of Information and Communication,Guilin University of Electronic Technology,Guilin 541004,China;School of Information and Communication,Guilin University of Electronic Technology,Guilin 541004,China)
出处
《计算机工程与设计》
北大核心
2022年第8期2221-2227,共7页
Computer Engineering and Design
基金
国家自然科学基金项目(62001133、61661017、61967005、61362021)
广西科技基地和人才专项基金项目(桂科AD19110060)
广西自然科学基金项目(2017GXNSFBA198212)
广西无线宽带通信与信号处理重点实验室基金项目(GXKL06200114)
认知无线电教育部重点实验室基金项目(CRKL150103)
桂林电子科技大学研究生创新基金项目(2019YCXS020)。
关键词
灾难性遗忘
贝叶斯增量学习
不确定性
学习率
权值选择策略
catastrophic forgetting
Bayesian incremental learning
uncertainty
learning rate
weight selection strategy