动态约束下可重构模块机器人分散强化学习最优控制

Decentralized reinforcement learning optimal control for time varying constrained reconfigurable modular robot

基于ction-critic-identifier(ACI)与RBF神经网络,提出了一种外界动态约束下的可重构模块机器人分散强化学习最优控制方法,解决了存在强耦合不确定性的模块机器人系统的连续时间非线性最优控制问题。文中将机器人动力学模型描述为一个交联子系统的集合,基于连续时间MDPs性能指标,结合ACI与RBF神经网络,对子系统最优值函数,最优控制策略及总体不确定项进行辨识,使系统满足HJB方程下的最优条件,从而使可重构模块机器人子系统渐进跟踪期望轨迹,跟踪误差收敛且有界。采用Lyapunov理论对系统稳定性进行证明,数值仿真验证了所提出的分散控制策略的有效性。

Based on Action-Critic-Identifier （ACI） and Radial Basis Function （RBF） neural network, a novel decentralized reinforcement learning optimal control method for time varying constrained reconfigurable modular robot is presented. The continuous time nonlinear optimal control problem of strongly coupled uncertainty robotic system is solved. The dynamics of the robot is described as a synthesis of interconnected subsystems. As a precondition to the continuous-time MDPs performance indicators, the optimal value function, optimal control policy and global uncertainty of the subsystems are estimated combing with ACI and RBF network. The optimal conditions of HJB equation with regard to the subsystem are satisfied, so that the reconfigurable modular robot system can track the desired trajectory in a short time and the estimation error can converge to zero in finite time. The stability of the system is confirmed by Lyapunov theory. Simulations are performed to illustrate the effectiveness of the proposed decentralized control scheme.