基于变分自动编码器的动态主题模型

Dynamic topic model based on variational autoencoder

传统动态主题模型的后验分布推断需要复杂的推理过程,仅模型假设的细微变化就需要重新进行推断,时间成本较高,制约了模型的可变性和通用性。为了提高动态主题模型的性能,提出了基于变分自动编码器融合动态因子图进行推断的动态主题模型。该模型对变分下界进行再参数化,生成一个下界估计器,将隐变量转换为一组辅助参数,使得新的参数不依赖于变分参数,用标准随机梯度下降法直接优化变分目标,同时融合动态因子图对状态空间模型进行建模,弱化推断的概率特性,简化优化过程,实现有效的推断。结果表明,提出的模型不仅保证了准确性,而且其简化模型有效降低了推断的时间成本,从而为动态主题模型能有效应用于复杂的时间场景提供更多可能。

The posterior distribution of traditional dynamic topic model requires complex reasoning process,and a small change in model assume will require re-deduction,meanwhile with high time cost,which restricts the variability and generality of the model.A dynamic topic model based on variational autoencoder fusing with dynamic factor graph for inference is proposed in order to improve the performance of dynamic topic model.The model makes a reparameterization trick to evidence lower bound to generate a lower estimator,and converts the hidden parameters to a group of auxiliary parameters,which makes new parameters not depend on variational parameters;standard stochastic gradient descent method can be available to variational objective function directly.At the same time,integrating the dynamic factor graph on modeling the state space model weakens the probabilistic of the model,simplifies the optimization process,and makes effective inference.The experimental results show that this model guarantees the accuracy,and the simplified model reduces the time cost effectively,which will provide more possibilities for dynamic topic model to be applied to complex time scenarios effectively.