基于二叉级联结构的并行极速学习机算法

Parallel Extreme Learning Machines Based on Binary Cascade Architecture

为解决因庞大的矩阵存储和计算,ELM(Extreme Learning Machines)难以应用到大规模、高维数据集的问题,提出一种基于"分而治之"策略的并行极速学习机算法。该算法利用二叉级联结构,将大规模数据集分派到多个计算节点上,并行地更新单隐层前馈网络的输出权值,且能有限步地单调收敛到最小二乘解。实验结果表明,该算法不仅泛化性能优异,并且具有非常高的加速比和并行效率。

ELM（Extreme Learning Machines） always works inefficiently on large-scale and high-dimen- sion datasets for huge memory and computation costs. We prepose a novel parallel algorithm for training ELM quickly based on ＂divide and conquer＂ strategy. It dispatches large-scale datasets to a cluster of computing nodes by utilizing special binary cascade architecture,and then updates weights of SLFN（Single Hidden I.ayer Feed-forward Neural Network） in parallel. Theoretical analysis proves that the new al gorithm converges to the best least-square solution monotonously with finite steps. Preliminary experi mental results show that the new algorithm has good generalization ability, excellent speedup ratio and parallel efficiency.