一种基于MapReduce的并行PSO-BP神经网络算法

Parallel PSO-BP Neural Network Algorithm Based on MapReduce

为了提高BP神经网络算法的分类准确率和运行时间效率,利用PSO算法和并行化设计的思想,提出了Hadoop平台下基于MapReduce的PSO优化BP神经网络的并行化设计及实现方法。利用PSO算法优化BP神经网络的初始权值和阈值,提高算法分类准确率;采用MapReduce并行编程模型实现算法的并行化处理,解决了BP神经网络在处理大规模样本数据集时存在的硬件开销和通信开销大的问题。选用SUN Database场景图像库构造了5个不同规模的数据集,通过与传统的串行PSO-BP神经网络算法实验对比,并行化的PSO-BP神经网络算法分类准确率达92%左右,系统效率在0.85左右,在处理大规模数据集时具有明显的优越性。

To improve the time efficiency and the accuracy rate of classifying results of BP neural network,parallel PSO-BP neural network algorithm is proposed using the idea of PSO algorithm andparallelization on the Hadoop platform.PSO algorithm is used to optimize the initial weights andthresholds of BP neural network to improve the accuracy rate of classifying results;MapReduce parallelprogramming model is adopted to realize the parallel processing of the algorithm,which solves theproblem of the hardware overhead and communication overhead when BP neural network deals with largescale sample data sets.5different scale data sets are constructed by using SUN Database.Comparedwith traditional serial PSO-BP neural network algorithm,the accuracy rate of classifying results ofparallel PSO-BP neural network algorithm is about92%and the system efficiency is about0.85,whichhas obvious both in time and the classified results in processing larger data sets.