基于深度强化学习与扩展卡尔曼滤波相结合的交通信号灯配时方法

Traffic signal timing method based on deep reinforcement learning and extended Kalman filter

深度Q学习网络(DQN)因具有强大的感知能力和决策能力而成为解决交通信号灯配时问题的有效方法,然而外部环境扰动和内部参数波动等原因导致的参数不确定性问题限制了其在交通信号灯配时系统领域的进一步发展。基于此,提出了一种DQN与扩展卡尔曼滤波(EKF)相结合(DQN-EKF)的交通信号灯配时方法。以估计网络的不确定性参数值作为状态变量,包含不确定性参数的目标网络值作为观测变量,结合过程噪声、包含不确定性参数的估计网络值和系统观测噪声构造EKF系统方程,通过EKF的迭代更新求解,得到DQN模型中的最优真实参数估计值,解决DQN模型中的参数不确定性问题。实验结果表明:DQN-EKF配时方法适用于不同的交通环境,并能够有效提高车辆的通行效率。

The deep Q-learning network(DQN)has become an effective method to solve the traffic signal timing problem because of its strong perception and decision-making ability.However,in the field of traffic signal timing systems,the problem of parameter uncertainty caused by external environment disturbance and internal parameter fluctuation limits its further development.Based on this,a traffic signal timing method combining DQN and extended Kalman filter(DQN-EKF)is proposed.In this method,the uncertain parameters of the estimated network are taken as the state variables,and the target network values with uncertain parameters are taken as the observed variables.The EKF system equation is constructed by combining the process noise,the estimated network values with uncertain parameters and the system observation noise.The optimal estimation of the parameters in the DQN model is obtained through the iterative updating of the EKF Uncertainty.The experimental results show that the DQN-EKF timing algorithm is suitable for different traffic environments and can effectively improve the traffic efficiency of vehicles.