隐私计算环境下深度学习的GPU加速技术综述

A Review of GPU Acceleration Technology for Deep Learning in Plaintext and Private Computing Environments

随着深度学习技术的不断发展,神经网络模型的训练时间越来越长,使用GPU计算对神经网络训练进行加速便成为一项关键技术.此外,数据隐私的重要性也推动了隐私计算技术的发展.首先介绍了深度学习、GPU计算的概念以及安全多方计算、同态加密2种隐私计算技术,而后探讨了明文环境与隐私计算环境下深度学习的GPU加速技术.在明文环境下,介绍了数据并行和模型并行2种基本的深度学习并行训练模式,分析了重计算和显存交换2种不同的内存优化技术,并介绍了分布式神经网络训练过程中的梯度压缩技术.介绍了在隐私计算环境下安全多方计算和同态加密2种不同隐私计算场景下的深度学习GPU加速技术.简要分析了2种环境下GPU加速深度学习方法的异同.

With the continuous development of deep learning technology,the training time of neural network models is getting longer and longer,and using GPU computing to accelerate neural network training has increasingly become a key technology.In addition,the importance of data privacy has also promoted the development of private computing technology.This article first introduces the concepts of deep learning,GPU computing,and two privacy computing technologies,secure multiparty computing and homomorphic encryption,and then discusses the GPU acceleration technology of deep learning in plaintext environment and private computing environment.In the plaintext environment,the two basic deep learning parallel training modes of data parallelism and model parallelism are introduced,two different memory optimization technologies of recalculation and video memory swapping are analyzed,and gradient compression in the training process of distributed neural network is introduced.technology.This paper introduces two deep learning GPU acceleration techniques:Secure multiparty computation and homomorphic encryption in a privacy computing environment.Finally,the similarities and differences of GPUaccelerated deep learning methods in the two environments are briefly analyzed.