面向人工智能物联网的分布式训练通信优化策略

The Optimization Strategy of Distributed Training Communication for Artificial Intelligence Internet of Things

为了提高物联网中人工智能模型的训练效率,提出了一种基于阈值划分的梯度选择策略,以解决分布式训练过程中节点间通信开销高的问题.通过研究主流的模型来分析梯度参数的分布.根据梯度的分布特点,该策略将梯度分为关键区域和稀疏区域.结合梯度分布的信息熵,该策略确定合理的阈值对分区内的梯度值进行选择,仅选择大于阈值的梯度值参与训练.实验评估结果表明,该策略可以有效地减少传输的参数量,并确保了模型训练的准确性.

In order to improve the training efficiency of artificial intelligence model in the Internet of things,a gradient selection strategy based on threshold division is proposed to solve the problem of high communication overhead between nodes in the distributed training process.Firstly,the distribution of gradient parameters is analyzed by studying the mainstream models.According to the distribution characteristics of the gradient,the strategy divides the gradient into key areas and sparse areas.Combined with the information entropy of gradient distribution,the strategy determines a reasonable threshold to select the gradient value in the partition,and only select the gradient value greater than the threshold to participate in the training.The experimental evaluation results show that this strategy can effectively reduce the amount of parameters transmitted and ensure the accuracy of model training.