基于深度强化学习与多参数域随机化的水下机械手自适应抓取研究

Deep Reinforcement Learning and Multi-Parameter Domain Randomization Based Underwater Adaptive Grasping Research for Underwater Manipulator

以水下机械手自主作业的应用需求为背景,针对水下机械手动力学参数时变、工作环境复杂、传感器限制、控制精度低等问题,基于强化学习与多参数域随机化理论提出一个具有通用性的水下机械手作业框架。首先,建立基本的机器人强化学习控制系统,然后采用多参数域随机化方法增强强化学习训练策略的稳定性与策略迁移效果,包括机械手动力学参数、水动力参数、状态空间与动作空间的噪声和延时等;最后,将训练的策略分别迁移到一个新的机器人仿真环境与一款真实的工作级水下机械手上进行实验。大量实验验证了本文所提方法的有效性,为未来真实海域自主作业奠定了基础。

This study proposes a general control system for underwater manipulation, which combines deep reinforcement learning and domain randomization for autonomous underwater manipulation of underwater manipulators. First,a reinforcement learning-based robot control system is established.Subsequently,multi-parameter domain randomization is used to improve the policy robustness and transferring effectiveness,including parameters of manipulator dynamics,hydrodynamic parameters,and noise and delay of state and action spaces. Finally,the trained policy is deployed on a new simulation environment and real underwater arm. The experimental results verify the validity of the proposed method and lay a foundation for autonomous manipulation in the real deep-sea environment in the future.