MapReduce在科学计算中的研究与改进

The research and improvement of MapReduce in scientific computing

针对Haloop模型不能实现各个计算节点的通信和Twister模型出现大量的数据重叠,提出了以下的改进:在Hadoop模型中增加各个节点的通信机制和缓冲机制。具体的实施如下:首先,在Map函数中引入了一个参数M来区分科学计算中的四类算法;其次,并将经常用的函数封装成适配器;再者,静态数据声明成保护类型并存放在缓冲池中。在文章的最后利用Hadoop做的相关实验,实验表明:随着计算节点数的增多,其加速比是越来越大的。

Against the problem of a large numberdata coverage in Twister and not communicating with different computing nodes, Made the following improvements： to increase each node communication mechanism and buffering mechanism in the Hadoop .The specific embodiment is as follows：First of all, this paper introduced the parameter of M in the Map function in order to distinguish four categories algorithms of scientific computing . Secondly, functions which are frequently used were packaged into the adapter , At the same time, the static data was decared as the type of protection in order to protect data safety. Finally, this paper cited the examples in the last of the paper and did a few of associated experiments. The experiment showed that with the increase of the number of computing nodes, the speedup is growing.