基于强化学习的固定翼飞机姿态控制方法

Reinforcement learning based attitude controller design

研究基于强化学习的飞机姿态控制方法,控制器输入为飞机纵向和横向状态变量以及姿态误差,输出为升降舵和副翼偏转角度指令,实现不同初始条件下飞机姿态角快速响应,同时避免使用传统PID控制器和不同飞行状态下的参数调节.根据飞机姿态变换特性,通过设置分立的神经网络模型提高算法收敛效率.为贴近实际的固定翼飞机控制,仿真基于JSBSim的F-16飞机空气动力学模型,利用OpenAI gym搭建强化学习仿真环境,以任意角速度、角度和空速作为初始条件,对姿态控制器中的动作网络和评价网络进行训练.仿真结果表明,基于强化学习的姿态控制器响应速度快,动态误差小,并能避免大过载等边界条件.

This article presents an attitude controller based on reinforcement learning(RL).The inputs of the actor network are states of attitude angle,angular rates etc,where the output is the angle control command of elevator and aileron,achieving the rapid response of the attitude angle with variable initial conditions,avoiding the application of the conventional PID controller and the parameter adjustment.According to the states transfer characteristics,by setting the splitting neural network model,the efficiency of algorithms is improved.In order to be close to the actual fixed-wing aircraft model,the simulation is based on the JSBSim F-16 aerodynamic model,using the OpenAI gym to build the simulation environment for reinforcement learning.With arbitrary angular speed,angle,and airspeed as initial conditions,the actor and critic networks are trained.The simulation results show that the RL based attitude controller has faster response and less dynamic error compared with the conventional PID controller.