基于Spark的分布式机器人强化学习训练框架

Training Framework of Distributed Robot Reinforcement Learning Based on Spark

强化学习能够通过自主学习的方式对机器人难以利用控制方法实现的各种任务进行训练完成,有效避免了系统设计人员对系统建模或制定规则。然而,强化学习在机器人开发应用领域中训练成本高昂,需要花费大量时间成本、硬件成本实现学习训练,虽然基于仿真可以一定程度减少硬件成本,但对类似Gazebo这样的复杂机器人训练平台,仿真过程工作效率低,数据采样耗时长。为了有效解决这些问题,针对机器人仿真过程的平台易用性、兼容性等方面进行优化,提出一种基于Spark的分布式强化学习框架,为强化学习的训练与机器人仿真采样提供分布式支持,具有高兼容性、健壮性的特性。通过实验数据分析对比,表明本系统框架不仅可有效提高机器人的强化学习模型训练速度,缩短训练时间花费,且有助于节约硬件成本。

Through autonomous learning, reinforcement learning can train robots to complete various tasks that are difficult for them to implement with control methods, and this can effectively avoid system designers from systemic modeling or rules making. However, the training cost of reinforcement learning in the field of robot development and application is high, and it takes a large amount of time cost and hardware cost to realize learning and training. Although the hardware cost can be reduced to some extent based on simulation, for the complicated robot training platform such as Gazebo, the working efficiency of simulation process is low, and it takes a long time for data sampling. In order to effectively solve these problems, a distributed reinforcement learning framework based on Spark is put forward, which optimizes the usability and compatibility of platform of robot simulation process, offers distributed support for the training of reinforcement learning and robot simulation sampling, and has the characteristics of high compatibility and robustness. Through analyzing and contrasting the experimental data, the system framework can not only effectively improve the training speed of reinforcement learning model of robot and shorten the training time, but also help with the saving of hardware cost.