基于强化学习的变电站巡检路径规划算法

Substation Inspection Path Planning Algorithm Based on Reinforcement Learning

针对变电站巡检机器人需要在多种复杂变电站环境下进行快速部署和执行检测点遍历巡检任务的需求,提出了一种基于强化学习和多层神经网络的巡检路径规划算法。在强化学习框架下,提出了基于多层感知器的状态-行为预测模型,以实现根据在线感知和历史经验信息对当前最优的行为决策进行预测。通过对算法训练过程中的奖励函数进行有效设计,使用近端策略优化(Proximal Policy Optimization, PPO)对模型进行训练,诱导机器人同时实现检测点遍历和障碍物规避的目标。在主流仿真环境平台Open AI Gym上搭建仿真环境并进行实验验证。验证结果表明,所提出算法能够在多种类型变电站完成检测点遍历巡检路径规划任务。

In accordance with the need of substation inspection robots to quickly deploy and perform inspection point traversal inspection tasks in a variety of complex substation environments, this paper proposes an inspection path planning algorithm based on reinforcement learning and multi-layer neural network. Firstly, under the framework of reinforcement learning, a state-behavior prediction model based on multi-layer perceptron was proposed to predict the current optimal behavior decision based on the current online perception and historical experience information. Then, a continuous reward function was proposed. Proximal Policy Optimization(PPO) was used to induce the robot to achieve the goal of detection point traversal and obstacle avoidance at the same time. Finally, the simulation environment was built on Open AI Gym and experiments are performed. The verification results show that the algorithm can complete the inspection path planning tasks of traversal detection points and obstacle avoidance for various types of substations.