基于Ray并行分布式框架的深度强化学习计算平台

Deep Reinforcement Learning Computing Platform Based on Ray Parallel Distributed Framework

近些年随着深度强化学习的不断发展,其训练成本也在不断增加,然而传统的训练平台大部分是基于顺序执行训练,不仅训练时间长、硬件成本高昂,且数据采样也非常困难。为了解决这些问题,本文中提出了一种基于Ray并行分布式架构的深度强化学习计算平台(RRLP),平台利用固定资源预算进行异步并行训练,兼容机器人仿真环境,不仅可以节约硬件资源,还可以加快采样与训练速度提高效率。通过实验可知基于Ray并行分布式架构的深度强化学习计算平台优于传统的计算平台,且有一定稳定性和可扩展性。

In recent years, with the continuous development of deep reinforcement learning, its training cost is also increasing. However, most of the traditional training platforms are based on sequential execution of training, which not only takes a long time to train, the hardware cost is high, but also data sampling is very difficult. In order to solve these problems, this paper proposes a deep reinforcement learning computing platform(RRLP) based on the Ray parallel distributed architecture. The platform uses a fixed resource budget for asynchronous parallel training and is compatible with the robot simulation environment. It can not only release hardware resources, but also speed up sampling and training to improve efficiency. Experiments show that the deep reinforcement learning computing platform based on the Ray parallel distributed row architecture is superior to the traditional computing platform, and has certain stability and scalability.