摘要
未来空战正朝着无人化、自主化方向发展,自主空战决策方法是未来空战的重要支撑手段之一。传统空战决策方法由于维度限制,存在无法处理连续动作与远视决策的问题。基于Actor-Critic方法提出空战连续决策的统一方法架构,依据空战训练经验对状态空间、动作空间、奖励及训练科目进行合理设计,测试多种连续动作空间强化学习算法在高不确定性空战场景下的学习效果并进行可视化验证。结果表明:基于本文提出的方法架构,可以实现连续动作下的远视价值寻优,智能体可以在复杂空战态势下做出最优决策,对随机机动飞行目标有较高的击杀率,且空战机动轨迹具有较高的合理性。
The future air warfare is developing in the unmanned and autonomous direction.The autonomous air warfare decision-making methods are one of the important support methods in future.Due to dimensional limitations,traditional air combat decision-making methods cannot handle continuous action and long-sighted decision-making problems.Based on the Actor-Critic method,a unified architecture for continuous decision-making in air combat is proposed in this paper.Combining air combat training experience,the state space,action space,reward and training subjects are rationally designed,and a variety of continuous action space reinforcement learning algorithms are tested in high uncertainty.The learning effect in the air combat scenario is visually verified.The results show that:based on the method architecture proposed in this paper,long-sighted value optimization under continuous actions can be realized,the agent can make optimal decisions in complex air combat situations,and has a high kill rate against random maneuvering flying targets.And the air combat maneuver trajectory is highly reasonable.
作者
单圣哲
杨孟超
张伟伟
高传强
SHAN Shengzhe;YANG Mengchao;ZHANG Weiwei;GAO Chuanqiang(School of Aeronautics,Northwestern Polytechnical University,Xi’an 710072,China;93995 Unit of the Chinese People's Liberation Army,Xi’an 710306,China)
出处
《航空工程进展》
CSCD
2022年第5期47-58,共12页
Advances in Aeronautical Science and Engineering
基金
国防科技重点实验室基金(6142219190302)。
关键词
自主空战
强化学习
人工智能
深度神经网络
autonomous air combat
reinforcement learning
artificial intelligence
deep neural network