基于改进Q-learning算法移动机器人局部路径研究

Research on Local Path of Mobile Robot Based on Improved Q-learning Algorithm

针对局部路径规划时因无法提前获取环境信息导致移动机器人搜索不到合适的路径,以及在采用马尔可夫决策过程中传统强化学习算法应用于局部路径规划时存在着学习效率低下及收敛速度较慢等问题,提出一种改进的Q-learn-ing(QL)算法。首先设计一种动态自适应贪婪策略,用于平衡移动机器人对环境探索和利用之间的问题;其次根据A*算法思想设计启发式学习评估模型,从而动态调整学习因子并为搜索路径提供导向作用;最后引入三阶贝塞尔曲线规划对路径进行平滑处理。通过Pycharm平台仿真结果表明,使得改进后的QL算法所规划的路径长度、搜索效率及路径平滑性等特性上都优于传统Sarsa算法及QL算法,比传统Sarsa算法迭代次数提高32.3%,搜索时间缩短27.08%,比传统QL算法迭代次数提高27.32%,搜索时间缩短17.28%,路径规划的拐点大幅度减少,局部路径优化效果较为明显。

In local path planning,the mobile robot can't find a suitable path because it can't get the environmental information in advance,and there are some problems such as low learning efficiency and slow convergence speed when the traditional reinforce-ment learning algorithm in markov decision processes is applied to local path planning.In this paper,an improved Q-learning(QL)algorithm is proposed.Firstly,a dynamic adaptive greedy strategy is designed to balance the problems between mobile robots'explo-ration and utilization of the environment.Secondly,a heuristic learning evaluation model is designed according to the idea of A*al-gorithm,so as to dynamically adjust learning factors and provide guidance for searching paths.Finally,the third-order Bezier curve programming is introduced to smooth the path.The simulation results on Pycharm platform show that the path length,search efficien-cy and path smoothness planned by the improved QL algorithm are superior to those of the traditional Sarsa algorithm and QL algo-rithm.Compared with the traditional Sarsa algorithm,the iteration times are increased by 32.3%,the search time is shortened by 27.08%,the iteration times are increased by 27.32%,the search time is shortened by 17.28%,the inflection point of path planning is greatly reduced,and the local path optimization effect is obvious.