无信号交叉口处基于深度强化学习的智能网联车辆运动规划

Intelligent connected vehicle motion planning at unsignalized intersections based on deep reinforcement learning

为了兼顾无信号交叉口处智能网联车辆通行效率和舒适性要求,提出基于深度强化学习的车辆运动规划算法.结合时间卷积网络(TCN)和Transformer算法构建周围车辆行驶意图预测模型,通过多层卷积和自注意力机制提高车辆运动特征捕捉能力;利用双延迟深度确定性策略梯度(TD3)强化学习算法构建车辆运动规划模型,综合考虑周围车辆行驶意图、驾驶风格、交互风险以及自车舒适性等因素设计状态空间和奖励函数以增强对动态环境的理解;通过延迟策略更新和平滑目标策略提高算法稳定性,实时输出期望加速度.实验结果表明,所提运动规划算法能够根据周围车辆的行驶意图实时感知潜在的交互风险,生成的运动规划策略满足通行效率、安全性和舒适性要求,且对不同风格的周围车辆和密集交互场景均有良好的适应能力,不同场景下成功率均高于92.1%.

A vehicle motion planning algorithm based on deep reinforcement learning was proposed to satisfy the efficiency and comfort requirements of intelligent connected vehicles at unsignalized intersections.Temporal convolutional network(TCN)and Transformer algorithms were combined to construct the intention prediction model for surrounding vehicles.The multi-layer convolution and self-attention mechanisms were used to improve the capability of capturing vehicle motion feature.The twin delayed deep deterministic policy gradient(TD3)reinforcement learning algorithm was employed to build the vehicle motion planning model.Taking the driving intention of surrounding vehicle,driving style,interaction risk,and the comfort of ego vehicle into consideration comprehensively,the state space and reward functions were designed to enhance understanding the dynamic environment.Delaying the policy updates and smoothing the target policies were conducted to improve the stability of the proposed algorithm,and the desired acceleration was output in real-time.Experimental results demonstrated that the proposed motion planning algorithm can perceive the real-time potential interaction risk based on the driving intention of surrounding vehicles.The generated motion planning strategy met the requirements of the efficiency,safety and comfort.It showed excellent adaptability to different styles of surrounding vehicles and dense interaction scenarios,and the success rates exceeded 92.1%in various scenarios.