多步 R 学习算法(英文)

Incremental Multi Step R Learning 

目的 讨论平均准则下控制马氏链的强化学习算法，在事先未知状态转移矩阵及报酬函数的条件下，通过试凑法寻找使得长期每阶段期望平均报酬最大的最优控制策略． 方法 结合平均报酬问题的一步学习算法和即时差分学习算法，提出了一种多步强化学习算法——— Ｒ（ λ） 学习算法． 结果与结论 新算法使得已有的 Ｒ 学习算法成为其λ＝ ０ 时的特例． 同时它也是折扣报酬 Ｑ（λ） 学习算法到平均报酬问题的自然推广． 仿真结果表明λ取中间值的 Ｒ（ λ） 学习算法明显优于一步的 Ｒ 学习算法．

Aim To investigate the model free multi step average reward reinforcement learning algorithm. Methods By combining the R learning algorithms with the temporal difference learning (TD( λ ) learning) algorithms for average reward problems, a novel incremental algorithm, called R( λ ) learning, was proposed. Results and Conclusion The proposed algorithm is a natural extension of the Q( λ) learning, the multi step discounted reward reinforcement learning algorithm, to the average reward cases. Simulation results show that the R( λ ) learning with intermediate λ values makes significant performance improvement over the simple R learning.