一种基于双经验池优先采样的深度强化学习算法

A Deep Reinforcement Learning Algorithm Based on Double Experience Memory and Prioritized Experience Replay

智能体在游戏、机器人控制、自动驾驶和自然语言处理等领域有着广泛应用。然而,稀疏奖励问题成为智能体学习和探索的困难之一。文章提出了改进算法,采用双经验池存储经验样本,并融入优先经验采样以提高采样效率。同时,对奖励函数进行重构,细分为多段奖励,以引导智能体学习。实验结果表明,改进算法优于传统DQN(Deep Q-Network)算法和同策略的A2C(Advantage Actor-Critic)算法,有效应对了稀疏奖励问题,并提高了智能体的学习效率。在经典Cartpole游戏环境中进行的实验验证了改进算法的优越性。

Intelligent agents have been widely applied in various fields,such as gaming,robot control,autonomous driving,and natural language processing.However,the problem of sparse rewards has become a challenging issue for learning and explora-tion in these domains.This paper proposes an improved algorithm that utilizes dual experience replay buffers and incorporates prioritized experience sampling to enhance the efficiency of data sampling.Furthermore,the reward function is restructured into multiple segments,providing guiding rewards to facilitate the learning process of the intelligent agent.Experimental results demonstrate that the proposed algorithm outperforms traditional Deep Q-Network(DQN)and Advantage Actor-Critic(A2C)al-gorithms,effectively addressing the challenges posed by sparse reward settings and significantly improving the learning effi-ciency of the intelligent agent.The effectiveness of the improved algorithm is validated through experiments conducted in the classic Cartpole gaming environment.