基于动态先验的LDA模型消息传递算法

BELIEF PROPAGATION OF LATENT DIRICHLET ALLOCATION BASED ON DYNAMIC PRIORI PARAMETERS

变分贝叶斯、吉布斯采样和消息传递是求解潜在狄利克雷分配(LDA)模型的三种主要近似推理算法,消息传递算法在效率和准确率上都明显优于其他两种。为了获得高可解释性的潜在语义空间,提出在迭代过程中动态调整先验参数的消息传递算法,使用加入伽马先验的固定点迭代方法自动学参数,同时探索对称先验以及非对称先验对模型泛化能力及文本分类准确率的影响。实验结果表明提出的动态非对称先验算法改进了模型的泛化能力,提高了文本分类的准确率。

There are three main approximate inference methods to seek the solution of latent Dirichlet allocation（ LDA） model： the variational Bayes,the Gibbs sampling and the belief propagation. Belief propagation algorithm is obviously competitive in both efficiency and accuracy to other two. For finding the latent semantic space with high interpretability,this paper proposes the belief propagation algorithm which dynamically adjusts priori parameters during iterations. It automatically learns the parameters by the fixed point iteration method with Gamma priori added. Meanwhile,we explore the effect of symmetric priori and asymmetric priori on the generalisation ability of model and the accuracy of text classification. Experimental results show that the proposed dynamic asymmetric priori algorithm improves the generalisation ability of model as well as raises the accuracy of text classification.