仿生机器人运动步态控制:强化学习方法综述

Locomotion gait control for bionic robots:a review of reinforcement learning methods

仿生机器人是一类典型的多关节非线性欠驱动系统,其步态控制是一个非常具有挑战性的问题。对于该问题,传统的控制和规划方法需要针对具体的运动任务进行专门设计,需要耗费大量时间和精力,而且所设计出来的控制器往往没有通用性。基于数据驱动的强化学习方法能对不同的任务进行自主学习,且对不同的机器人和运动任务具有良好的通用性。因此,近年来这种基于强化学习的方法在仿生机器人运动步态控制方面获得了不少应用。针对这方面的研究,本文从问题形式化、策略表示方法和策略学习方法3个方面对现有的研究情况进行了分析和总结,总结了强化学习应用于仿生机器人步态控制中尚待解决的问题,并指出了后续的发展方向。

The bionic robot is a typical multi-joint,nonlinear,underactuated system,for which locomotion gait control is of much challenge.For this problem,traditional control and planning methods need to be carefully designed for specific locomotion tasks,which takes a lot of time and efforts,yet lacks generality.On the contrary,data-driven reinforcement learning method can autonomously learn the controller for different locomotion tasks,and it presents the advantage of good generality for different bionic robots and locomotions.Therefore,in recent years,this reinforcement learning-based method has been widely used in the field of bionic robots to construct various locomotion gait controllers.In this paper,the current research status of reinforcement learning-based methods for the locomotion control of bionic robots is comprehensively analyzed,respectively from the following three aspects:formulation of the problem,policy representation,and policy learning.Finally,the problems to be solved in the field are and summarized,and the possible future research directions are provided.