航空器智能引导机动决策奖励重塑方法

Reward Shaping for Intelligent Maneuver Decision Generation in Aircraft Guidance

针对使用深度强化学习进行航空器智能引导研究中存在的飞行轨迹质量差、训练效率低等问题,对应用于机动决策生成的奖励重塑方法进行了研究。首先,构建了航空器引导机动决策生成的深度强化学习模型;其次,从指令连续性和相对姿态两个角度设计了奖励重塑函数,并证明了使用重塑函数前后的策略一致性;最后,在不同类型目的地场景中进行了仿真实验。仿真结果表明:奖励重塑方法对航空器飞行轨迹质量和智能体训练效率有明显的提升。使用本方法快速训练的智能体,可以准确、高效地生成机动决策,引导航空器完成任务。

A reward shaping approach applied to maneuver decision generation was proposed to solve the poor trajectory performance and low training efficiency problems in aircraft intelligent guidance using deep reinforcement learning.Firstly,a deep reinforcement learning model for aircraft guidance maneuver decision generation was built.Secondly,the reward shaping function was designed from the two aspects of instruction continuity and relative position,and the policy consistency using reward shaping was proved.Finally,simulations were carried out in different kinds of destination scenarios.Simulation results show that flight trajectory performance and agent training efficiency are significantly improved by using reward remodeling.The agent trained by the proposed method can generate maneuver decision accurately and efficiently and guide the aircraft to complete the guidance task.