最小状态变元平均奖赏的强化学习方法

Reinforcement learning algorithm based on minimum state method and average reward

针对采用折扣奖赏作为评价目标的Q学习无法体现对后续动作的影响问题,提出将平均奖赏和Q学习相结合的AR-Q-Learning算法,并进行收敛性证明。针对学习参数个数随着状态变量维数呈几何级增长的"维数灾"问题,提出最小状态变元的思想。将最小变元思想和平均奖赏用于积木世界的强化学习中,试验结果表明,该方法更具有后效性,加快算法的收敛速度,同时在一定程度上解决积木世界中的"维数灾"问题。

In allusion to the problem that Q-Learning,which was used discount reward as the evaluation criterion,could not show the affect of the action to the next situation,AR-Q-Learning was put forward based on the average reward and Q-Learning.In allusion to the curse of dimensionality,which meant that the computational requirement grew exponen-tially with the number of the state variable.Minimum state method was put forward.AR-Q-Learning and minimum state method were used in reinforcement learning for Blocks World,and the result of the experiment shows that the method has the characteristic of aftereffect and converges more faster than Q-Learning,and at the same time,solve the curse of di-mensionality in a certain extent in Blocks World.