基于MapReduce的并行PLSA算法及在文本挖掘中的应用

MapReduce Based Parallel Probabilistic Latent Semantic Analysis for Text Mining

PLSA(Probabilistic Latent Semantic Analysis)是一种典型的主题模型。复杂的建模过程使其难以处理海量数据,针对串行PLSA难以处理海量数据的问题,该文提出一种基于MapReduce计算框架的并行PLSA算法,能够以简洁的形式和分布式的方案来解决大规模数据的并行处理问题,并把并行PLSA算法运用到文本聚类和语义分析的文本挖掘应用中。实验结果表明该算法在处理较大数据量时表现出了很好的性能。

PLSA（（Probabilistie Latent Semantic Analysis） is a typical topic model. To enable a distributed computation of PLSA for the ever-increasing large datasets, a parallel PLSA algorithm based on MapReduce is proposed in this paper. Applied in text clustering and semantic analysis, the algorithm is demonstrated by the experiments for s its scalability in dealing with large datasets.