基于深度确定性策略梯度的智能车汇流模型

Traffic Merging Model for Intelligent Vehicle Based on Deep Deterministic Policy Gradient

采用离散动作空间描述速度变化的智能车汇流模型不能满足实际车流汇入场景的应用要求,而深度确定性策略梯度(DDPG)结合策略梯度和函数近似方法,采用与深度Q网络(DQN)相同的网络结构,并使用连续动作空间对问题进行描述,更适合描述智能车速度变化。为此,提出一种基于DDPG算法的智能车汇流模型,将汇流问题转化为序列决策问题进行求解。实验结果表明,与基于DQN的模型相比,该模型的收敛速度较快,稳定性和成功率较高,更适合智能车汇入车辆场景的应用。

Traffic merging models for intelligent vehicle that use discrete action space to describe changing speed cannot meet the application requirements of actual traffic merging scenarios.Deep Deterministic Policy Gradient(DDPG),which integrates policy gradient with function approximation methods and adopts the same network structure as Deep Q-Network(DQN),uses continuous action space for problem description.So DDPG is more suitable for describing the changing speed of intelligent vehicles.On this basis,this paper proposes a traffic merging model for intelligent vehicles based on the DDPG algorithm,reducing the traffic merging problem to a sequence decision problem to be resolved.Experimental results show that compared with DQN-based models,the proposed model has a faster convergence speed,higher reliability and a higher success rate,which means it is more applicable to traffic merging scenarios of intelligent vehicle.