基于深度强化学习的固定翼飞行器六自由度飞行智能控制

Fixed-Wing Aircraft 6-DOF Flight Control Based on Deep Reinforcement Learning

发展无人机先进飞行控制技术是各军事强国不断追求的目标,而智能飞行控制更是无人机空战智能决策的基础.强化学习提供了一种能自适应并优化、独立于模型、广泛适用于各种对象的控制机设计技术范式,是实现智能飞行控制的有效途径.相较于三自由度运动,六自由度运动描述更加接近于飞行器真实的飞行,但是由于动力学模型的非线性以及气动力的复杂性,固定翼飞行器六自由度强化学习控制的实现更加困难.基于面向多维连续状态输入、多维连续动作输出的深度强化学习技术,针对全尺寸固定翼飞行器发展从飞行状态到舵面/推力控制的端到端六自由度一体化智能控制机,避免传统控制器中航迹/姿态控制双回路结构的人为分割设计,通过引入偏航角误差作为控制机输入,实现近零侧滑的稳定巡航飞行,同时发展的控制机具有一定的可拓展性,为后续空战智能决策研究奠定基础.

The development of advanced autonomous flight control for unmanned aerial vehicles is the goal of military powers,and the intelligent flight control is the foundation for the intelligent air combat of the vehicle.Reinforcement learning provides a novel and general controller design paradigm that is adaptive,optimized,model-free and widely applicable,and it is a promising way for the intelligent control.In contrast to the 3 degree-of-freedom(DOF)flight,the 6-DOF motion better describes the aircraft real flight.However,due to the nonlinearity of the dynamics and complexity of the aerodynamics,the implementation of the 6-DOF flight intelligent control for the fixed-wing aircraft is difficult.Based on the multiple continuous states input and multiple continuous action output deep reinforcement learning,the end-to-end integrated intelligent control for the cruise flight,directly from the vehicle flight states to the aero-surfaces and thrust control,is developed for the full-scaled fixed-wing aircraft.It avoids the artificiall trajectory and attitude loop separation in the classic controller design.By introducing the error of the yaw angle as the input,the stable cruise flight with nearly zero sideslip is achieved and the developed controller is applicable to different cruise tasks,which is useful for the future research on the air combat intelligent decision-making.