Behavioral decision-making at urban intersections is one of the primary difficulties currently impeding the development of intelligent vehicle technology.The problem is that existing decision-making algorithms cannot ...Behavioral decision-making at urban intersections is one of the primary difficulties currently impeding the development of intelligent vehicle technology.The problem is that existing decision-making algorithms cannot effectively deal with complex random scenarios at urban intersections.To deal with this,a deep deterministic policy gradient(DDPG)decision-making algorithm(T-DDPG)based on a time-series Markov decision process(T-MDP)was developed,where the state was extended to collect observations from several consecutive frames.Experiments found that T-DDPG performed better in terms of convergence and generalizability in complex intersection scenarios than a traditional DDPG algorithm.Furthermore,model-agnostic meta-learning(MAML)was incorporated into the T-DDPG algorithm to improve the training method,leading to a decision algorithm(T-MAML-DDPG)based on a secondary gradient.Simulation experiments of intersection scenarios were carried out on the Gym-Carla platform to verify and compare the decision models.The results showed that T-MAML-DDPG was able to easily deal with the random states of complex intersection scenarios,which could improve traffic safety and efficiency.The above decision-making models based on meta-reinforcement learning are significant for enhancing the decision-making ability of intelligent vehicles at urban intersections.展开更多
基金supported in part by the Beijing Municipal Science and Technology Project(No.Z191100007419010)Automobile Industry Joint Fund(No.U1764261)of the National Natural Science Foundation of China+1 种基金Shandong Key R&D Program(No.2020CXGC010118)Key Laboratory for New Technology Application of Road Conveyance of Jiangsu Province(No.BM20082061706)。
文摘Behavioral decision-making at urban intersections is one of the primary difficulties currently impeding the development of intelligent vehicle technology.The problem is that existing decision-making algorithms cannot effectively deal with complex random scenarios at urban intersections.To deal with this,a deep deterministic policy gradient(DDPG)decision-making algorithm(T-DDPG)based on a time-series Markov decision process(T-MDP)was developed,where the state was extended to collect observations from several consecutive frames.Experiments found that T-DDPG performed better in terms of convergence and generalizability in complex intersection scenarios than a traditional DDPG algorithm.Furthermore,model-agnostic meta-learning(MAML)was incorporated into the T-DDPG algorithm to improve the training method,leading to a decision algorithm(T-MAML-DDPG)based on a secondary gradient.Simulation experiments of intersection scenarios were carried out on the Gym-Carla platform to verify and compare the decision models.The results showed that T-MAML-DDPG was able to easily deal with the random states of complex intersection scenarios,which could improve traffic safety and efficiency.The above decision-making models based on meta-reinforcement learning are significant for enhancing the decision-making ability of intelligent vehicles at urban intersections.
