风险概率准则下的非平稳马氏决策过程

Nonstationary Markov Decision Processes with Risk Probability Criteria

本文研究一类非平稳离散马氏决策过程的风险概率最小化问题,其中转移概率和奖励函数随时间变化.与现有文献中的期望报酬/成本准则不同,本文考虑最小化系统在首次到达某个目标集之前获得的总报酬未能达到给定利润目标的概率.在合理的假设条件下,我们建立了相应的最优方程序列,验证了最优风险函数序列是最优方程序列的唯一解,并证明了最优马氏策略的存在性.

This paper considers a risk probability minimization problem for nonstationary discrete-time Markov decision processes,in which the transition probabilities and the reward functions depend on time.Different from the expected reward/cost criteria in the existing literature,the optimality performance here is to minimize the probability that the total rewards do not reach a given profit goal until the first passage time to some target set.Under mild reasonable conditions,we establish the corresponding optimality equations,verify that the sequence of the optimal risk functions is the unique solution to the optimality equations,and prove the existence of an optimal Markov policy.