基于强化学习的通信受限环境多无人机协同策略

Cooperative Strategy of Multiple Unmanned Aerial Vehicles in Limited Communication Environment Based on Reinforcement Learning

随着人工智能技术的发展,空域无人作战正由“单平台遥控”向“多平台协同”转变。多无人机协同作战任务具有非完全信息、通信受限、高实时、强动态等特点,给协同决策生成带来巨大挑战。针对通信受限环境中的多无人机协同决策问题,提出一种基于动态层级网络通信架构的通信强化学习协同策略,该策略能够显著减少无人机集群间的通信次数,同时准确传递其决策需要的信息,从而得到较优协同策略。针对多无人机协同围捕的典型任务场景,基于OpenAI平台对所提出的算法进行了仿真验证。结果表明,与传统强化学习算法相比,提出的通信强化学习策略可以显著减少无人机间的通信次数,同时在一定程度上避免潜在的信息欺骗问题。完成任务需要的平均通信次数相比于传统两两通信结构减少约77%,为实现通信受限环境中的多无人机协同任务提供技术支撑。

With the development of artificial intelligence technology,airspace unmanned combat is changing from"single-platform remote control"to"multi-platform cooperation".Multi-UAV cooperative task has the characteristics of incomplete information,limited communication,high real-time,strong dynamic,etc.,which brings great challenges to the collaborative decision-making generation.This paper proposes a communication reinforcement learning cooperation strategy based on dynamic hierarchical network communication architecture for multi-UAV cooperative decision-making in communication constrained environment.This strategy can significantly reduce the communication times between UAVs,while accurately transmitting the information needed for decision-making,so as to obtain a better cooperation strategy.In this paper,the proposed algorithm is simulated based on OpenAI platform for typical task scenarios of multi-UAV cooperative capture.The results show that compared with the traditional reinforcement learning algorithm,the communication reinforcement learning strategy proposed in this paper can significantly reduce the communication times between UAVs,and avoid the potential information deception problem to some extent.The average communication times required to complete the task are reduced by about 77%compared with the traditional two-way communication structure.It provides technical support for the realization of multi-UAV cooperative task in communication limited environment.