摘要
As a parallel programming model, Map-Reduce is used for distributed computing of massive data. Map-Reduce model encapsulates the details of parallel implementation, fault-tolerant processing, local computing and load balancing, etc., provides a simple but powerful interface. In case of having no clear idea about distributed and parallel programming, this interface can be utilized to save development time. This paper introduces the method of using Hadoop, the open-source Map-Reduce software platform, to combine PCs to carry out scalable parallel computing. Our experiment using 12 PCs to compute N-body problem based on Map-Reduce model shows that we can get a 9.8x speedup ratio. This work indicates that the Map-Reduce can be applied in scalable parallel computing.
As a parallel programming model, Map-Reduce is used for distributed computing of massive data. Map-Reduce model encapsulates the details of parallel implementation, fault-tolerant processing, local computing and load balancing, etc., provides a simple but powerful interface. In case of having no clear idea about distributed and parallel programming, this interface can be utilized to save development time. This paper introduces the method of using Hadoop, the open-source Map-Reduce software platform, to combine PCs to carry out scalable parallel computing. Our experiment using 12 PCs to compute N-body problem based on Map-Reduce model shows that we can get a 9.8x speedup ratio. This work indicates that the Map-Reduce can be applied in scalable parallel computing.
基金
Project supported by the Shanghai Leading Academic Discipline Project(Grant No.J50103)
the National High-Technology Research and Development Program of China(Grant No.2009AA012201)
the Major Technology R&D Program of Shanghai(Grant No.08DZ501600)
the Science and Technology Pillar Project of Jiangxi(Grant No.2010BGB00604)
