混合交通流环境下基于MSIF-DRL的网联自动驾驶车辆换道决策模型

Lane-changing decision-making model for connected and automated vehicles based on MSIF-DRL in a mixed traffic environment

现有的网联自动驾驶车辆(Connected and Automated Vehicles,CAV)换道决策模型鲁棒性较差,存在安全隐患,且单纯依赖自车信息、较小范围内的感知信息,难以在CAV与人工驾驶车辆(Human-Driven Vehicles,HDV)混行的环境中推断出最优动作.综合考虑感知信息、自车以及车-车通信(Vehicle-to-Vehicle,V2V)范围内上、下游CAV信息,提出一种混合交通流环境下集成多源信息融合的深度强化学习(Multi-Source Information Fusion Deep Reinforcement Learning,MSIF-DRL)端到端网联自动驾驶换道决策模型.首先,构建含有多源信息的状态空间,并为不同信息分配权重;其次,通过编码网络将各种动态信息编码到高维特征空间,进行信息融合得到特征图;然后,将其扁平化送入拥有优先经验回放机制的竞争双深度Q网络中,进行动作的选择和评估;最后,分别设计适用于主线、匝道CAV的奖励函数引导所提MSIF-DRL模型解决高速公路合流区驾驶场景中CAV的自由以及强制换道问题.基于SUMO软件在各种仿真条件下进行实验,将所提出的MSIF-DRL换道决策模型与现有换道模型进行比较,验证其有效性和优越性.研究结果表明:相较于现有模型,所提MSIF-DRL模型在各种仿真条件下均拥有最高的奖励值、换道成功率、合流成功率、平均行车速度、舒适性以及最低的碰撞风险,其中换道成功率、合流成功率、平均行车速度最大分别提升了29.17%、27.71%、17.43%;随着渗透率的提高,该模型在处理混合交通流环境下CAV的换道决策问题时拥有更强的性能和鲁棒性.

Existing lane-changing decision-making models for Connected and Automated Vehicles(CAV)lack robustness,pose safety concerns and rely primarily on the ego-vehicle’s information and limited sensor data,making it challenging to infer optimal actions in mixed environments with CAV and Human-Driven Vehicles(HDV).To this end,this paper introduces an end-to-end lane-changing decision model for CAV in mixed traffic environments based on Multi-Source Information Fusion Deep Reinforcement Learning(MSIF-DRL).It considers information from the sensor data,ego-vehicle,and Vehicle-to-Vehicle(V2V)communication within the CAV’s upstream and downstream range.Firstly,a state space containing multi-source information is constructed,with the information from different vehicles assigned weights.Secondly,various dynamic multi-source information is en-coded into a high-dimensional feature space through the encoding network for information fusion to ob-tain the feature map.Then,the feature map is flattened and fed into the dueling deep double Q net-work with prioritized experience replay for action selection and evaluation.Finally,a reward function applicable to the mainline and on-ramp CAV is designed to guide the proposed MSIF-DRL model,solving the discretionary and mandatory lane change problems of CAV in freeway merging scenarios.Through simulation experiments using SUMO software under various conditions,the proposed MSIF-DRL model is compared with existing models to verify its effectiveness.The experimental re-sults show that,compared to the existing models,the proposed MSIF-DRL model achieves the high-est rewards,successful lane-change rate,successful merge rate,average driving speed,ride comfort,and the lowest collision risk under different simulation scenarios.The proposed model increases the successful lane change rate,successful merge rate,and average driving speed by 29.17%,27.71%,and 17.43%,respectively.Furthermore,as the penetration rate increases,the MSIF-DRL model ex-hibits enhanced performance and robustness in addressing CAV’s lane-changing decisions in mixed traffic environments.