近似强化学习算法研究综述

Review of Research on Approximate Reinforcement Learning Algorithms

强化学习用于解决无模型情况下的优化决策问题,是实现人工智能的重要技术之一,但传统的表格型强化学习方法难以处理具有大规模、连续空间的控制问题。近似强化学习受到函数逼近思想的启发,对价值函数或策略函数参数化表示,通过参数优化间接获得最优行为策略,在视频游戏、棋类对抗及机器人控制等领域应用效果显著。基于此,对近似强化学习算法的研究现状与应用进展进行了梳理和综述。介绍了近似强化学习相关的基础理论;分类总结了近似强化学习的经典算法及一些相应的改进方法;概述了近似强化学习在机器人控制领域的研究进展,并总结了当前面临的若干主要问题,为后续的研究提供参考。

Reinforcement learning(RL)is one of the most important techniques for artificial intelligence(AI). However,traditional tabular reinforcement learning is difficult to deal with control problems with large scale or continuous space.Approximate reinforcement learning is inspired by the idea of function approximation to parameterize the value function or strategy function, and obtains the optimal strategy indirectly through parameter optimization. It has been widely used in video games, Go game, robot control, etc. and obtained remarkable performance. In view of this, this paper reviews the research status and application progress of approximate reinforcement learning algorithms. Firstly, the basic theory of approximate reinforcement learning is introduced. Then the classical algorithms of approximate reinforcement learning are classified and expounded, including some corresponding improvement methods. Finally, the research progress of approximate reinforcement learning in robotics is summarized, and some major problems are summarized to provide reference for future research.