适于进化算法的迭代式MapReduce框架

Iteration MapReduce framework for evolution algorithm

MapReduce模块化的编程大大降低了分布式算法的实现难度,但同时也限制了它的应用范围。介绍了MapReduce的基本结构及其实现迭代算法的缺陷,并针对基于MapReduce进化算法效率低下的问题,在对MapReduce的计算框架进行研究的基础上提出了一种适用于进化算法的迭代式MapReduce计算框架。描述了迭代式MapReduce计算框架的实现需求及其具体实现,提出并证明了异常机制的可行性,且在公有的Hadoop云计算平台上对提出的框架进行了验证。实验结果表明,基于迭代式MapReduce计算框架的并行遗传算法在算法的加速比上与基于MapReduce的并行遗传算法相比有较大的提高。

Modular programming of MapReduce greatly simplifies the implementation difficuhy of distributed programming; however, its application scope is limited. In view of that MapReduce cannot be used to solve iteration algorithm, a new iteration MapReduce framework was proposed for evolutionary algorithm based on the study of MapReduce framework. The basic structure of the MapReduce was introduced, and the defects in implementing iteration algorithm were pointed out. The realization requirements and implementation of the proposed MapReduce framework were introduced, and the feasibility of abnormal mechanism was proposed and verified. At last, the new MapReduce framework was verified on Hadoop. The experimental results show that the parallel genetic algorithm based on the iteration MapReduce framework has higher speedup than that of MapReduce framework.