基于深度强化学习的多机协同空战方法研究

Research on Multi-aircraft Cooperative Air Combat Method Based on Deep Reinforcement Learning

多机协同是空中作战的关键环节,如何处理多实体间复杂的协作关系、实现多机协同空战的智能决策是亟待解决的问题.为此,提出基于深度强化学习的多机协同空战决策流程框架(Deep-reinforcement-learning-based multi-aircraft cooperative air combat decision framework,DRL-MACACDF),并针对近端策略优化(Proximal policy optimization,PPO)算法,设计4种算法增强机制,提高多机协同对抗场景下智能体间的协同程度.在兵棋推演平台上进行的仿真实验,验证了该方法的可行性和实用性,并对对抗过程数据进行了可解释性复盘分析,研讨了强化学习与传统兵棋推演结合的交叉研究方向.

Multi-aircraft cooperation is the key part of air combat,and how to deal with the complex cooperation relationship between multi-entities is the essential problem to be solved urgently.In order to solve the problem of intelligent decision-making in multi-aircraft cooperative air combat,a deep-reinforcement-learning-based multiaircraft cooperative air combat decision framework(DRL-MACACDF)is proposed in this paper.Based on proximal policy optimization(PPO),four algorithm enhancement mechanisms are designed to improve the synergistic degree of agents in multi-aircraft cooperative confrontation scenarios.The feasibility and practicability of the method are verified by the simulation on the wargame platform,and the interpretable review analysis of the antagonistic process data is carried out,and the cross research direction of the combination of reinforcement learning and traditional wargame deduction is discussed.