基于PPO的球形机器人目标跟随研究

Research on target following of spherical robot based on PPO

球形机器人由于其优异的运动性能、出色的地形适应能力和防侧翻的特性,被广泛应用于水下探测、岸滩巡检等需要适应复杂环境的场景。然而球形机器人系统模型具有欠驱动、非线性的特点,运动控制问题复杂,在复杂应用环境下难以可靠跟随目标。为此,提出了一种基于近端策略优化(PPO)算法的球形机器人目标跟随方法。该方法基于深度强化学习理论,在球形机器人动力学模型的基础上,设计了简单高效的动作空间和表征完善的状态空间。并且为提高目标跟随方法的鲁棒性,该方法在奖励函数中引入人工势场,以使目标始终保持在机器人视野中心。仿真结果表明,所提方法能够满足既定场景的跟随需求,球形机器人使用该方法可以对随机运动目标进行可靠跟随。

Spherical robots have become a popular choice for exploring underwater environments,inspecting shores and beaches,and handling other complex scenarios due to their exceptional motion performance,remarkable terrain adaptability,and anti-rollover characteristics.However,the spherical robot system model has inherent features of underdrive and nonlinearity,making motion control a challenging task,especially in complex environments where following targets reliably can be difficult.To address this issue,we propose a new method for following targets using the proximal policy optimization(PPO)algorithm.Our method utilizes deep reinforcement learning theory to design the corresponding state and action spaces based on the spherical robot dynamics model.We introduce an artificial potential field in the reward function to keep the target centered in the robot’s field of view,thereby enhancing the robustness of the target following method.Our simulation results demonstrate that the proposed method meets the requirements of the given scenario,allowing the spherical robot to follow a randomly moving target reliably.