一种合作Markov决策系统

A Cooperation Markov Decision Process System

在机器学习中,强化学习是一个重要的研究领域。Markov决策过程(MDP)是强化学习的重要基础,在一般的Markov决策系统中,只考虑一个智能体的学习演化。但目前诸多问题中只考虑单个智能体的学习演化有一定的局限性,越来越多的应用中都涉及到多个智能体。进而引入一种带有两个智能体的联合Markov决策系统(CMDP),该系统适用于两个智能体之间合作决策的学习演化。智能体之间存在合作或博弈两种类型,文中重点研究合作类型的CMDP,在此类学习模型中,智能体交替执行行为,以社会价值作为求优准则,寻找最优策略对(π*0,π*1),共同完成目标任务。进一步给出了在联合Markov系统中寻找最优策略对的算法,其根本任务是寻找一个最优策略对(π*0,π*1),形成一个合作系统CMDP(π*0,π*1),且系统模型可以进一步扩充到多个智能体的联合决策系统。

Reinforcement learning is an important research area in machine learning.Then Markov decision process(MDP)is the important basics in reinforcement learning.In the usual Markov decision system,only one agent’s learning evolution is considered.Among many problems,only the learning evolution of a single agent is considered,which has certain limitations,but the actual application involves multiple agents.For the reason,a cooperation Markov decision process(CMDP)with two agents is introduced,which is suitable for the learning evolution of cooperation decision between two agents.The focuses of the research is the cooperative CMDP.In this kind of learning model,the agent alternately performs behaviors,seeks the optimal pair of strategies(π*0,π*1)with social value optimization criteria and accomplishes the target tasks together.Researching the algorithm for finding the optimal strategy pair(π*0,π*1),which is to find an optimal strategy pair and form an evolutionary system CMDP(π*0,π*1).In addition,this system model can also be extended to the joint decision system of multiple agents.