基于改进的深度强化学习多智能体协作方法

Multi-agent collaboration method based on improved deep reinforcement learning

针对多智能体深度强化学习在值函数拟合过程中未充分考虑智能体之间的作用关系,且动作大概率随机,导致迭代试错过程的数据浪费、协作效率低、收敛速度慢等问题,提出了一种在协作中的平均权重机制和改进的探索策略。首先,利用平均深度Q学习网络(DQN)在多智能体的值函数策略网络中设计一种权重结构,减小智能体间的不利影响;其次,改进探索策略,利用欧氏距离提高智能体的探索效率与策略协作性,增大系统跳出局部最小点的能力。通过多个场景实验的结果表明,所提方法提高了多智能体的学习能力和学习效率。

Aiming at the problem that multi-agent deep reinforcement learning,the action relationship between the agents is not fully considered in value function fitting process,and the action is random with high probability,which leads to data waste of iterative process of trial error,low collaboration efficiency,slow convergence speed,and so on,an average weight mechanism in collaboration and an improved exploration strategy are proposed.Firstly,the average deep Q learning network(DQN)is used to design a weight structure in the multi-agent value function strategy network to reduce the adverse influence among agents.Secondly,the exploration strategy is improved by using Euclidean distance,which not only improves exploration efficiency of the agent and strategic collaboration,but also increases the ability of the system to jump out of the local minimum point.The results of experiments in multiple scenarios show that the proposed method improves the learning ability and learning efficiency of multi-agents.