一种基于Spark的高光谱遥感图像分类并行化方法

A kind of hyperspectral image classification parallel method based on Spark

为了实现大数据量遥感图像的分类,提出了一种Spark平台下高光谱遥感图像稀疏表示分类并行化方法 PSCSRC,首先设计五元组形式的中间数据存储结构,其次在每次迭代中只是将稀疏矩阵中与中间矩阵各分块对应的子矩阵分发到各节点,这就减少了中间数据Shuffle和数据传递的时间消耗。实验结果表明,PSCSRC分类方法与SCSRC分类方法相比,在保证分类精度的同时,执行速度有了明显的提升。

In order to achieve the classification of large amounts of hyperspectral image , proposed a hyperspectral sparse representation classification parallel method on Spark , designing a 5-tuple intermediate data storage structure and then distributing sub-matrix of spare matrix corresponded to each block of intermediate matrices to each node in each iteration ,which reduce time consumption caused by intermediate data Shuffle and data transmission .Experiment results show that execution speed of sparse representation classification parallel method under Spark platform has been significantly improved.