基于DQN的自动驾驶机器人速度跟踪控制

Vehicle Speed Tracking Control with a Robotic Driver Based on Deep Q-Network

由于汽车传动模型的复杂性、延迟性和踏板的死区特性,现有的基于传统控制理论和车辆模型的方法很难达到理想的控制效果。为解决这个问题,构建了一种基于DQN的速度跟踪算法,基于马尔可夫性设计了状态空间、动作空间,并根据超差规则设计奖赏函数。通过批量真车转鼓试验对所建立的速度跟踪算法进行了验证,结果表明:算法模型可有效控制踏板进行速度跟踪;从零开始,只需4~5轮训练即可满足超差数要求;与基于传统控制理论的方案相比,具有超差数更少、速度更平稳、无需专业人员调试等优势。

With the complexity and delay of the vehicle transmission model and the dead zone of the pedal,the existing control methods based on traditional control theory and vehicle model are difficult to achieve the ideal control effect.To solve this problem,a vehicle speed tracking algorithm based on Deep Q-Network(DQN)is constructed,state space and action space are designed based on Markov property,and reward function is designed according to out-of-tolerance rules.Experiments are conducted on dozens of cars,and results show that established algorithm can effectively control the robotic driver for speed tracking.From scratch,it only needs 4-5 episodes of training to meet the requirements.Compared with classic control methods,the proposed method has a smoother speed and fewer speed errors,and does not require experts’tuning.