预测状态表示模型的复位算法

An Algorithm for Resetting PSR Models

预测状态表示(Predictive State Representations,PSRs)是用于解决局部可观测问题的有效方法.然而,现实环境中,通过样本学习得到的PSR模型不可能完全准确.随着计算步数的增多,利用PSR模型计算得到的预测向量有可能越来越偏离其真实值,进而导致PSR模型的预测精度越来越低.文中提出了一种PSR模型的复位算法.通过使用判别分析方法确定系统所处的PSR状态,文中所提算法可对利用计算获取的预测向量复位,从而提高PSR模型的准确性.实验结果表明,采用复位算法的PSR模型在预测精度上明显优于未采用复位算法的PSR模型,验证了所提算法的有效性.

Predictive State Representations （PSRs） have been proposed as an alternative to partially observable Markov decision processes （POMDPs） to model dynamical systems. Although POMDPs and PSRs provide general frameworks for solving partially observable problems, in real world applications, when the PSR model of a system is learned from samples, it will almost certainly result in an inaccurate PSR model. Therefore the prediction vector calculated using this model may progressively drift farther and farther away from reality, which will result in lower prediction accuracy of the PSR model. This paper describes an algorithm for resetting the learned PSR models. First, for the inaccurate PSR model, the PSR state is identified using discriminant function analysis, then the calculated prediction vector can be reset for the purpose of improving the veracity of the PSR model. The algorithms with and without resetting the PSR model are compared, empirical results show that in case of the obtained PSR model＇s prediction quality, the algorithm with resetting the prediction vector has better prediction accuracy than the algorithm without resetting the prediction vector, which proves the effectiveness of the proposed algorithm.