风险敏感度激励学习的广义平均算法

Risk-Sensitive Reinforcement Learning Algorithms With Generalized Average Criterion

提出了一种新的算法.这个算法通过潜在地牺牲控制策略的最优性来获取其鲁棒性.这是因为,如果在理论模型与实际的物理系统之间存在不匹配,或者实际系统是非静态的,或者控制动作的可使用性随时间的变化而变化时,那么鲁棒性就可能成为一个十分重要的问题.主要工作是给出了一组逼近算法和它们的收敛结果.利用广义平均算子来替代最优算子max(或min),对激励学习中的一类最重要的算法——动态规划算法——进行了研究,并讨论了它们的收敛性,目的就是为了提高激励学习算法的鲁棒性.同时使用了更具一般性的风险敏感度性能评价体系,发现基于动态规划的学习算法中的一般结论在这种体系之下并不完全成立.

A new algorithm which immolates optimality of conrail poficies potentially to obtain the robusticity of solutions is proposed. The robusticity of solutions may become a very important property for a learning system due to when there exists non-mactching between theory models and practical physical system, or the practical system is not static, or availability of a control action will change along with variety of time. The main contribution is that a set of approximation algorithms and its convergence results will be given. Applying generalized average operator instead of the general optimal operator max （ or min） a class of important learning algorithm, dynamic programming algorithm were studied, and their convergence from theoretic point of view was disease. The purpose is to improve robusticity of reinforcement learning algorithms theoretically.