基于共享GPU的深度学习训练性能实证研究

AN EMPIRICAL STUDY ON TRAINING PERFORMANCE OF DEEP LEARNING BASED ON SHARED-GPU

深度学习应用的训练过程是计算密集型的,它通常依靠图形处理单元(Graphics Processing Unit, GPU)来加速训练过程。然而深度学习开发框架往往会独占GPU,造成计算资源的浪费。针对该问题,该实证研究对两个深度学习应用共享GPU训练的可行性进行讨论,系统地分析了有代表性的深度学习模型的静态和运行时特性,展示了共享GPU训练两个模型时,不同的模型组合和特征对整体性能的影响。根据实验结果所总结的原则可以作为提高调度效率和改善GPU云资源利用率的指导方针。

The training process of deep learning(DL) application is computation-intensive. It often relies on Graphics Processing Unit(GPU) to accelerate the training process. However, DL frameworks tend to monopolize the GPU, resulting in a waste of computing resource. In view of the problem, this paper conducts an empirical study on exploring the feasibility of executing two DL applications on one shared-GPU. We systematically analyzed the static and runtime characters of several typical DL models, and showed the influence of different model combinations and characteristics on the overall performance when shared-GPU trained two models. The principles summarized from experimental results can serve as guidelines to improve the scheduling efficiency and the utilization of GPU cloud resources.