一种基于强化学习的小库位自动平行泊车运动规划方法

A Motion Planning Method Based on Reinforcement Learning for Automatic Parallel Parking in Small Slot

为解决传统方法或基于强化学习的方法在狭小空间下平行泊车效率较低的问题,基于蒙特卡罗树搜索(MCTS)方法,同时规划倒车入库和库位内调整阶段。在MCTS过程中同时考虑纵向动作(速度)与横向动作(转向盘转角),引入模仿学习(IL),利用非线性规划的演示数据获得初始化策略神经网络,并使用强化学习(RL)对其进行改进,训练时间从20 h缩短到1 h,采用滑模控制器作为横向控制器来跟踪规划的路径,车辆运动方向可通过绑定在规划路径上的规划速度的方向确定。仿真验证和实车测试结果表明,该方法可同时规划倒车入库阶段和库位内调整阶段,位置误差可达5 cm,航向角误差可达0.5°。

In order to solve the problem of low efficiency for parallel parking in narrow space with traditional methods or methods based on reinforcement learning,this paper proposes a Monte Carlo Tree Search(MCTS)based method to plan simultaneously reversing and adjusting stages of parallel parking in tiny parking space.In the MCTS process,longitudinal action(velocity)and lateral action(steering wheel angle)are considered at the same time.Imitation Learning(IL)is introduced,which utilizes demonstration data from non-linear programming to get an initialized policy neural network and then uses RL to improve it,which reduces the training time from 20 h to 1 h.A sliding mode controller is adopted as lateral controller to track the planned path and the direction can be determined by the planned velocity profile bound on the planned path.The results of simulations and vehicle tests show that the proposed method can plan simultaneously 2 stages of parallel parking and complete the parking in tiny slot within 5 cm position error and 0.5°heading angle error.