基于大数据分析的人工神经网络分布式训练方法

Distributed Training Method of Artificial Neural Network Based on Big Data Analysis

为了降低人工神经网络训练时的复杂度并减少传统分布式训练方法的通信开销,提出了基于大数据分析的人工神经网络分布式训练方法。具体来讲,使用动态模型平均方法,仅在局部模型显著偏离全局模型时才对局部模型进行同步,因此与基于周期平均的分布式训练框架相比,减少了通信方面的不必要开销。实验部分,基于实际场景中的大型数据集和深度全卷积神经网络,证明了模型同步所需的通信时间明显缩短,且动态模型平均的方法可以达到与静态周期平均方法相当的精度,此外以证明其随着计算节点的增加而可横向扩展,这些夯实了本文方法的有效性。

In order to reduce the complexity of artificial neural network during the training and reduce the communication overhead of traditional distributed training methods,this paper proposes a distributed training method of artificial neural network based on big data analysis.Specifically,the dynamic model averaging method is used to synchronize the local model only when it deviates significantly from the global model.Therefore,compared with the distributed training framework based on period averaging,the unnecessary overhead in communication is reduced.In the experiment part,based on the actual scene of large data sets and depth of the convolution neural networks,proves that communication time required by the model synchronization is significantly shortened,and the dynamic model of average method can achieve the precision of the method to be equal to the static cycle average.Otherwise,it proves that the increase of computing nodes can scale out the end strengthen which shows the effectiveness of the proposed method.