隐马尔可夫模型估计的新方法及应用案例

A New Method about the Estimation of Hidden Markov Model and Its Application

隐马尔可夫模型广泛应用于经济、金融及大数据领域。目前,模型估计的主要方法是基于极大似然估计的Viterbi算法。本文从随机过程的常返理论出发,给出了隐马尔可夫模型参数估计的新方法。首先,利用从同一状态的观察值出发到固定点的首中时有相同分布的原理,给出隐状态个数的估计;再根据首中时数学期望与平稳分布的关系得到平稳分布和发射概率的估计;最后以上述方法为基础,完成了隐马尔可夫模型的两个应用研究:构建个性化推荐系统;揭示我国经济周期不同阶段间的转换规律。本文提出的新估计方法可以大幅减少计算复杂度,是Viterbi算法的有益补充。

Hidden Markov model is widely used in economics, finance and big data. The model estimation is the Viterbi algorithm based on the maximum likelihood estimation. In this paper, a new estimation method is given about hidden Markov models which is based on the recurrence theory of stochastic processes.Our method relies on the fact that the first hitting times to a fixed observation are identical distribution if starting points corresponds to the unique hidden state. Firstly, the number of hidden states is estimated. According to the relationship between the mathematical expectation of the first hitting time and the stationary distribution, the estimation of the emission probability is obtained. Furthermore, an estimate of the transfer matrix is obtained. Finally, applying the new method to the film recommendation system of the Grouplens Reach dataset, our method is better than other models;they also are used to study the regional system of business cycle in China since 2000. The new method proposed in this paper can greatly reduce the computational complexity and is a useful complement to the Viterbi algorithm.