航天器姿态稳定强化学习鲁棒最优控制方法

Reinforcement learning robust optimal control for spacecraft attitude stabilization

针对外部干扰力矩作用下的刚体航天器姿态稳定最优控制问题,提出了一种在线强化学习的智能鲁棒控制方法。该方法基于自适应动态规划框架,设计单Critic神经网络在线地学习无干扰作用的航天器的最优姿态控制律,并设计一种新的自适应律在线估计Critic神经网络的权值,实现了近似最优的控制性能。在学习的近似最优控制律的基础上,嵌入鲁棒控制量,形成鲁棒智能控制器,并应用Lyapunov理论证明了闭环姿态控制系统是一致最终有界稳定的,且Critic神经网络的权值估计误差是收敛的。相比于采用Actor-Critic神经网络结构的自适应动态规划方法,该方法一方面削弱了对持续激励条件的依赖,另一方面降低了计算复杂度,并保证了姿态稳定控制性能对外部干扰具有较强的鲁棒性。

The problem of optimal attitude stabilization control of rigid spacecraft despite external disturbances is in⁃vestigated.An online reinforcement learning-based intelligent and robust control approach is presented via the adap⁃tive dynamic programming technique.In this approach,a critic-only neural network is developed to learn the optimal control policy of the spacecraft attitude system with external disturbance.A new estimation law is synthesized to esti⁃mate the weights of that network online.The learned controller can achieve near-optimal control performance.Then,a robust control effort is designed and added into the learned controller to formulate an intelligent and robust controller.It is proven that the closed-loop attitude system obtained from the proposed controller is uniformly ultimately bounded and that the weight estimation error of the Critic NN is convergent by Lyapunov theory.Comparison with the traditional actor-critical neural network-based control schemes shows that with less computation complexity and great robustness to external disturbances,the proposed control approach is less dependent of the persistent excitation condition.Simu⁃lation results verify the superior control performance of the proposed approach.