在非时齐马氏决策过程中的动态贝叶斯网络研究

Research on Dynamic Bayesian Networks in Non Time Homogenous Markov Decision Process

提出了用动态贝叶斯网络(DBN)对非时齐马氏决策系统进行建模的改进方法,使动态贝叶斯网络能被更广泛地应用于各种复杂的真实系统中.该方法的基本思路是,将扩展后的隐藏变量引入DBN的演化过程来建立假设条件所要求的马尔可夫模型,给出从不完整的样本数据集以及存在隐藏变量时来学习DBN结构的算法,进而用贝叶斯概率统计方法对后来的时间片的充分统计因子进行估计,并通过当前已存在的和估计的充分统计因子对基于时间变化的转移概率进行学习,以解决假设条件要求的转移概率的时不变性.原理性分析和仿真实验结果也验证了改进方法的有效性.

For the purpose of the further wide application of dynamic Bayesian networks （DBNs） to many real complex systems, a new approach was presented to improve the modeling of the non time homogenous Markov decision systems with DBNs, in which the extended hidden variables were introduced into the evolutional process to build Markov models required by the hypothesis conditions, and a structure learning algorithm of DBNs was given from the incomplete data set when the extended hidden variables existed. The sufficient statistics of the subsequent time slices were estimated using Bayesian probability statistical method, and then the time-variant transition probabilities were learned using both current sufficient statistics and estimated sufficient statistics. The theoretical analysis and simulation results show that the proposed approach is valid.