数据分配策略优化的全条带遥感影像流水线并行处理

PARALLEL PROCESSING OF LONG STRIP REMOTE SENSING IMAGE BASED ON PIPELINE WITH DATA DISTRIBUTION STRATEGY OPTIMISATION

如何快速处理大区域覆盖的高分辨率全条带遥感影像是遥感应用急需解决的问题。提出分布式环境下全条带遥感影像并行处理模型。该模型采用了数据并行和流水线并行两级并行方法。首先综合考虑遥感数据及其处理算法的特殊性和分布式环境的异构性,提出一种新的数据分配策略,然后在计算节点引入流水线并行技术。实验结果显示,该模型不仅可以灵活进行复杂的数据处理,还能有效提高数据处理效率。在7个计算节点的环境下,相较于传统的串行处理方法,可获得24.32的加速比。

How to quickly process huge area-covering and high-resolution long strip remote sensing image is the problem of remote sensing application to be urgently resolved. Aiming at this problem,this paper proposes a parallel processing model for long strip image in distributed environment. The model adopts two-stage parallel approach with data parallel and pipeline parallel. First,we propose a new data distribution strategy by comprehensively considering the particularity of remote sensing data as well as its processing algorithm and the heterogeneous nature of distributed environment; then we introduce pipeline parallel technology to each computing node. Experimental results show that the model can flexibly carry out complicated data processing,it can effectively improve the efficiency of data processing as well. Under the condition of 7 computation nodes,it is able to acquire the speedup of 24. 32 compared with traditional serial processing method.