基于SAC的无人机自主导航方法研究

Exploring UAV autonomous navigation algorithm based on soft actor-critic

针对现有深度强化学习算法在无人机自主导航任务中面临环境局部可观且感知信息不足问题,基于非确定性策略SAC(soft actor-critic)强化学习算法对未知环境下的端到端无人机自主导航任务展开研究。具体而言,提出了一种基于记忆增强机制的策略网络,通过对历史记忆信息与当前的观测整合处理,提取观测数据的时序依赖关系,从而增强局部可观条件下的状态估计能力,避免算法陷入局部最优解;设计了非稀疏奖励函数以缓解强化学习策略在稀疏奖励条件下难以收敛的问题;在Airsim+UE4仿真平台进行了多个复杂场景的训练验证。实验结果表明,所提方法导航成功率比基准算法提高10%,平均飞行距离缩短21%,有效增强了无人机自主导航算法稳定性和收敛性。

The existing deep reinforced learning algorithms cannot see local environments and have insufficient perceptual information on UAV autonomous navigation tasks.The paper investigates the UAV′s autonomous navigation tasks in its unknown environments based on the nondeterministic policy soft actor-critic(SAC)reinforced learning model.Specifically,the paper proposes a policy network based on a memory enhancement mechanism,which integrates the historical memory information processing with current observations to extract the temporal dependency of the statements so as to enhance the state estimation ability under locally observable conditions and avoid the learning algorithm from falling into a locally optimal solution.In addition,a non-sparse reward function is designed to reduce the challenge of the reinforced learning strategy to converge under sparse reward conditions.Finally,several complex scenarios are trained and validated in the Airsim+UE4 simulation platform.The experimental results show that the proposed method has a navigation success rate 10%higher than that of the benchmark algorithm and that the average flight distance is 21%shorter,which effectively enhances the stability and convergence of the UAV autonomous navigation algorithm.