基于概率模型的动态分层强化学习

Dynamic hierarchical reinforcement learning based on probability model

为解决大规模强化学习中的"维度灾难"问题,克服以往学习算法的性能高度依赖于先验知识的局限性,本文提出一种基于概率模型的动态分层强化学习方法.首先基于贝叶斯学习对状态转移概率进行建模,建立基于概率参数的关键状态识别方法,进而通过聚类动态生成若干状态子空间和学习分层结构下的最优策略.仿真结果表明该算法能显著提高复杂环境下智能体的学习效率,适用于未知环境中的大规模学习.

To deal with the overwhelming dimensionality in the large-scale reinforcement-learning and the strong depen-dence on prior knowledge in existing learning algorithms,we propose the method of dynamic hierarchical reinforcement learning based on the probability model（DHRL--model）.This method identifies some key states automatically based on probability parameters of the state-transition probability model established based on Bayesian learning,then generates some state-subspaces dynamically by clustering,and learns the optimal policy based on hierarchical structure.Simulation results show that DHRL--model algorithm improves the learning efficiency of the agent remarkably in the complex environment,and can be applied to learning in the unknown large-scale world.