基于MapReduce技术的海量文本数据统计方法研究

Research by Massive Text Data Statistics Method Based on MapReduce

为了实现海量文本数据的有效发掘和统计,从文本分类与大数据并行处理两个方面着手,研究在海量文本数据挖掘的相关理论知识和MapReduce编程模型的相关技术支撑下的海量文本数据挖掘统计方法。在并行运算模型的开源实现Hadoop平台上,提出了一种简单、有效的文本分类方法——基于MapReduce的平均朴素贝叶斯文本分类算法。

In order to realize the effective exploration of massive text data and statistics, from two aspects of the text classification and data parallel processing, this paper explores relevant theoretical knowledge of massive text data related technology research and MapReduce programming model for mining statistical method in mining. On the Hadoop open source implementation platform of parallel computing model, a simple and effective text classification method is provided based on MapReduce of average naive Bayes text classification algorithm.