自主空战连续决策方法

Continuous Decision-making Method for Autonomous Air Combat

未来空战正朝着无人化、自主化方向发展,自主空战决策方法是未来空战的重要支撑手段之一。传统空战决策方法由于维度限制,存在无法处理连续动作与远视决策的问题。基于Actor-Critic方法提出空战连续决策的统一方法架构,依据空战训练经验对状态空间、动作空间、奖励及训练科目进行合理设计,测试多种连续动作空间强化学习算法在高不确定性空战场景下的学习效果并进行可视化验证。结果表明:基于本文提出的方法架构,可以实现连续动作下的远视价值寻优,智能体可以在复杂空战态势下做出最优决策,对随机机动飞行目标有较高的击杀率,且空战机动轨迹具有较高的合理性。

The future air warfare is developing in the unmanned and autonomous direction.The autonomous air warfare decision-making methods are one of the important support methods in future.Due to dimensional limitations,traditional air combat decision-making methods cannot handle continuous action and long-sighted decision-making problems.Based on the Actor-Critic method,a unified architecture for continuous decision-making in air combat is proposed in this paper.Combining air combat training experience,the state space,action space,reward and training subjects are rationally designed,and a variety of continuous action space reinforcement learning algorithms are tested in high uncertainty.The learning effect in the air combat scenario is visually verified.The results show that:based on the method architecture proposed in this paper,long-sighted value optimization under continuous actions can be realized,the agent can make optimal decisions in complex air combat situations,and has a high kill rate against random maneuvering flying targets.And the air combat maneuver trajectory is highly reasonable.