并行智能训练技术:挑战与发展

Parallel intelligent computing:development and challenges

近年来,以深度学习为代表的人工智能技术迅猛发展,深度学习模型和训练数据的规模均呈爆炸式增长,给智能模型训练系统带来了巨大挑战.随着高性能计算与人工智能的不断深度融合,并行智能训练技术成为大规模深度学习模型高效训练的主要方法.本文总结了并行智能训练的基本模式和关键技术,以及并行智能训练框架的发展现状,分析了并行智能训练技术和框架发展面临的挑战与发展趋势,简介了银河天璇并行智能训练框架的研究进展.

The development of artificial intelligence technology with the use of deep learning has gained momentum in recent years,resulting in a considerable increase in the scale of deep learning models and training data,and high-performance computing and artificial intelligence technologies have been continuously and deeply integrated.Parallel intelligent training has become the main method for the efficient training of large-scale deep learning models.In this work,we evaluate the basic methods and key technologies of parallel intelligent training,outline the development status of the parallel intelligent training framework,summarize the challenges and trends in developing the parallel intelligent technology and framework,and propose development schemes for a Merak parallel intelligent training framework.