基于Q学习的高超声速飞行器自抗扰控制研究

Research on Active Disturbance Rejection Control of Hypersonic Vehicle Based on Q-learning

为实现高超声速飞行器姿态自抗扰控制的参数整定,提出一种模糊Q学习算法。首先,采用强化学习中的Q学习算法来实现姿态自抗扰控制参数的离线闭环快速自适应整定;然后,根据模糊控制的思路,将控制参数划分为不同区域,通过设定奖励,不断更新Q表;最后,将训练好的Q表用于飞行器的控制。仿真结果表明,相对于传统的线性自抗扰控制(linear active disturbance rejection control,LADRC)和滑模控制,基于Q学习的LADRC省去了人工调试参数的繁琐过程,且仍具有良好的跟踪效果。蒙特卡罗仿真测试结果验证了基于Q学习的LADRC的鲁棒性。

A fuzzy Q-learning algorithm is proposed to adjust the parameters of attitude active disturbance rejection control for hypersonic vehicles.Firstly,the Q-learning algorithm in reinforcement learning is used to realize the fast off-line closed-loop adaptive tuning of attitude active disturbance rejection control parameters.Then,according to the idea of fuzzy control,the control parameters are divided into different areas,and the Q-table is constantly updated by setting rewards.Finally,the trained Q-table is used to control the aircraft.Compared with conventional linear active disturbance rejection control(LADRC)and sliding mode control,simulation results show that LADRC based on Q-learning saves the tedious process of manual parameter tuning,and still has good tracking performance.Monte Carlo simulation results verify the robustness of LADRC based on Q-learning.