稀疏奖励下基于情感的异构多智能体强化学习

Emotion-Based Heterogeneous Multi-agent Reinforcement Learning with Sparse Reward

在强化学习中,当处于奖励分布稀疏的环境时,由于无法获得有效经验,智能体收敛速度和效率都会大幅下降.针对此类稀疏奖励,文中提出基于情感的异构多智能体强化学习方法.首先,建立基于个性的智能体情感模型,为异构多智能体提供激励机制,作为外部奖励的有效补充.然后,基于上述激励机制,融合深度确定性策略,提出稀疏奖励下基于内在情感激励机制的深度确定性策略梯度强化学习算法,加快智能体的收敛速度.最后,在多机器人追捕仿真实验平台上,构建不同难度等级的稀疏奖励情景,验证文中方法在追捕成功率和收敛速度上的有效性和优越性.

In reinforcement learning,the convergence speed and efficiency of the agent are greatly reduced due to its inability to acquire effective experience in an sparse reward distribution environment.Aiming at this kind of sparse reward problem,a method of emotion-based heterogeneous multi-agent reinforcement learning with sparse reward is proposed in this paper.Firstly,the emotion model based on personality is established to provide incentive mechanism for multiple heterogeneous agents as an effective supplement to external rewards.Then,based on this mechanism,a deep deterministic strategy gradient reinforcement learning algorithm based on intrinsic emotional incentive mechanism under sparse rewards is proposed to accelerate the convergence speed of agents.Finally,multi-robot pursuit is used as a simulation experiment platform to construct sparse reward scenarios with different difficulty levels,and the effectiveness and superiority of the proposed method in pursuit success rate and convergence speed are verified.