发现和学习不可复位动态系统的预测状态表示的一种新算法

A New Algorithm for Discovery and Learning of Predictive State Representations in Dynamical Systems Without Reset

提出了一种发现和学习不可复位动态系统的预测状态表示的新算法.在证明系统的任意landmark均可作为系统的初始状态的基础上,利用发现的landmark确定系统在任意时间步所处的经历,然后采用蒙特卡罗方法估计任意经历下任意检验发生的概率,解决了在不可复位动态系统中,经历下检验发生的概率难以获取问题,进而发现和学习不可复位动态系统的预测状态表示.实验结果表明,本文算法获得的系统的预测状态表示在预测精度上明显优于suffix-history算法,验证了所提算法的有效性.

A new algorithm for discovery and learning of predictive state representations in dynamical systems without reset is proposed. With proving that any landmark can be used as the initial state, the discovered landmarks are used to identify the history at any time step in a continues data,then the conditional probability of any test at any history is estimated using Monte Carlo approaches, which efficiently solves the difficult problem of obtaining the conditional probability in dynamical systems without reset, thereby it is straightforward to discover and learn predictive state representations. The empirical results show that in case of the obtained predictive state representations＇ s prediction quality, our algorithm has better prediction accuracy than the suffix-history algorithm, which proves the effectiveness of the proposed algorithm.