基于多智能体深度强化学习的船舶协同避碰策略

Ship cooperative collision avoidance strategy based on multi-agent deep reinforcement learning

船舶避碰是智能航行中首要解决的问题,多船会遇局面下,只有相互协作,共同规划避碰策略,才能有效降低碰撞风险.为使船舶智能避碰策略具有协同性、安全性和实用性,提出一种基于多智能体深度强化学习的船舶协同避碰决策方法.首先,研究船舶会遇局面辨识方法,设计满足《国际海上避碰规则》的多船避碰策略.其次,研究多船舶智能体合作方式,构建多船舶智能体协同避碰决策模型:利用注意力推理方法提取有助于避碰决策的关键数据;设计记忆驱动的经验学习方法,有效积累交互经验;引入噪音网络和多头注意力机制,增强船舶智能体决策探索能力.最后,分别在实验地图与真实海图上,对多船会遇场景进行仿真实验.结果表明,在协同性和安全性方面,相较于多个对比方法,所提出的避碰策略均能获得具有竞争力的结果,且满足实用性要求,从而为提高船舶智能航行水平和保障航行安全提供一种新的解决方案.

Ship collision avoidance is the primary issue in intelligent navigation.In multi-ship encounters,only by collaborating and jointly planning collision avoidance strategies,the collision risk can be effectively reduced.In order to make the ship intelligent collision avoidance strategy collaborative,safe and practical,a ship collaborative collision avoidance decision method based on multi-agent deep reinforcement learning is proposed.Firstly,the method of identifying ship encounter situations is studied and a multi-ship collision avoidance strategy that satisfies the"International regulations for preventing collisions at sea"is designed.Secondly,by analysing the cooperation mode of multi-ship agents,a multi-ship agent cooperative collision avoidance decision-making model is constructed.The model uses the attention inference method to extract the key data that is helpful for collision avoidance decisions.And a memory driven experience learning method is designed to effectively accumulate interactive experience.In addition,the noise network and multi-head attention mechanism are introduced into the model to enhance decision-making and exploration capabilities of ship agents.Finally,on the experimental map and the real nautical chart,simulation experiments are carried out on the multi-ship encounter scenarios.The results show that in terms of collaboration and safety,compared with multiple comparison methods,competitive results are obtained and the practical requirements are met using the proposed method,which provides a new solution for improving theintelligent navigation of ships and ensuring navigation safety.