基于深度强化学习的无人船全覆盖路径规划

USV coverage path planning based on deep reinforcement learning

无人船使用传统“之”字形算法在不规则岛屿区域执行海上搜索任务时,无法实现全覆盖路径规划。针对该问题,文中提出一种将“之”字形算法和基于深度强化学习的无人船全覆盖路径规划算法框架相结合的混合算法,对大范围无障碍区域使用“之”字形算法,对存在障碍的小范围区域使用深度强化学习算法框架,并引入内在好奇心模块增强该算法框架的收敛速度。该算法框架将搜索区域的地图信息转换成矢量观测值并通过全连接层传递给智能体,从而训练出一个神经网络为无人船做规划决策,在满足规避障碍物的安全约束条件下实现任务区域的全覆盖。试验方面,通过Unity3D仿真平台搭建三维环境模型,验证该混合算法的可行性。结果表明,所提出的无人船全覆盖路径规划算法框架可在岛屿区域达到覆盖率100%,且路径较短。

The unmanned surface vehicle(USV)cannot achieve full coverage path planning when using traditional zigzag algorithm to perform maritime search task in irregular island area.On this basis,a hybrid algorithm combining zigzag algorithm and USV full coverage path planning algorithm framework based on deep reinforcement learning is proposed.The zigzag algorithm is used for a large range of barrier-free areas,the deep reinforcement learning algorithm framework is used for small areas with obstacles,and the internal curiosity module is introduced to enhance the convergence speed of the algorithm framework.This algorithm framework can be used to convert the map information of the search area into vector observations and transfer them to the agent by means of the full connection layer,thus training a neural network to make planning decisions for USV,and achieving full coverage of the task area under the security constraints of obstacle avoidance.In the aspect of the experiment,the feasibility of the hybrid algorithm was verified by building a 3D environment model on the Unity3D simulation platform.The results show that the proposed USV coverage path planning algorithm can achieve 100%coverage and has a short path in the island area.