支持鹏程系列开源大模型应用生态演化的可持续学习能力探索

Exploration of the continual learning ability that supports the application ecological evolution of the large-scale pretraining Peng Cheng series open source models

大规模预训练模型利用大规模语料以及多样化的预训练任务在自然语言处理领域取得了巨大的成功。随着大模型的逐步发展,大模型的可持续学习能力探索成为新的研究热点。主要介绍鹏程系列大模型持续学习的技术体系、应用实践以及面临的挑战,包括通过任务扩展、数据增量和知识推理的鹏程系列可持续学习技术体系,开源大模型鹏程·盘古多任务可持续学习和鹏程·通言大模型的可持续学习能力实践,大模型可持续学习过程中面临的词表更新、语义映射和知识冲突等挑战。

Large-scale pre-training models have achieved great success in the field of natural language processing by us-ing large-scale corpora and pre-training tasks.With the gradual development of large models,the continual learning abil-ity of large models has become a new research focus.The continual learning technology of the Peng Cheng series large models,the exploration of practice and the still facing challenges were mainly introduced,including the Peng Cheng se-ries continual learning technology through task expansion,data increment and knowledge reasoning,Peng Cheng PAN-GU multi-task continual learning and the practical exploration of the continual learning ability of the Peng Cheng TON-GYAN open source large model,the vocabulary update,semantic mapping and knowledge conflicts that the large model faces in the process of continual learning.