强化学习方法在移动机器人导航中的应用

Research on reinforcement learning and its application to mobile robot

路径规划是智能机器人关键问题之一,它包括全局路径规划和局部路径规划.局部路径规划是路径规划的难点,当环境复杂时,很难得到好的路径规划结果.这里将强化学习方法用于自主机器人的局部路径规划,用以实现在复杂未知环境下的路径规划.为了克服标准Q 学习算法收敛速度慢等缺点,采用多步在策略SARSA(λ)强化学习算法,讨论了该算法在局部路径规划问题上的具体应用.采用CMAC神经网络实现了强化学习系统,完成了基于CMAC神经网络的SARSA(λ)算法.提出了路径规划和沿墙壁行走两个网络的互相转换的方法,成功解决了复杂障碍物环境下的自主机器人的局部路径规划问题.仿真结果表明了该算法的有效性,同传统方法相比该算法有较强的学习能力和适应能力.

Path planning,both global and local, is one of key problems of intelligent robot. Local path planning is particularly challenging. It is difficult to obtain a good path in a complex environment. To overcome this,the reinforcement learning method was used for local path planning in a unknown and complex environment. Aiming at the slow convergent rate and other drawbacks of standard Q-learning, the multi-step on-policy SARSA(λ) reinforcement algorithm was adopted in the field of robot's local path planning and the related problems were discussed. The CMAC neural network was used to realize the reinforcement learning system. The SARSA(λ) algorithm was implemented by using the CMAC neural network. The switching method between the path planning network and the wall-following network was adopted to resolve local path planning of a autonomous mobile robot in the environment with complex obstacles. The simulation results demonstrate the efficiency of the algorithm. The algorithm has good (adaptation) and self-learning ability compare with other traditional methods.