平均报酬指标多步递推最小二乘即时差分学习

Average Reward Multi-Step Temporal-Difference Learning Using Recursive Least-Squares Methods

对非周期不可约Markov链上的线性函数近似平均报酬指标即时差分学习方法进行了研究.近似器由权值进行增量更新的固定特征函数线性加权组合构成,在对已有的算法进行比较分析的基础上,利用线性参数估计理论的有关成果,提出了基于值函数线性近似表示的平均报酬指标多步递推最小二乘即时差分强化学习算法,并给出了其一致收敛性证明.

Average reward temporal-difference learning of an irreducible aperiodic Markov chain based on linear function approximations is investigated. Approximations are comprised of linear combinations of fixed basis function whose weight are incrementally updated. On the basis of analyzing and investigating the exist algorithms,and using the linear parameter estimation theory, a new class of average reward multi-step temporal-difference learning algorithms based on linear function approximations and recursive least squares methods is proposed. A proof of uniform converge is presented.