三维交错网格有限差分地震波模拟的GPU集群实现

Accelerating 3D Staggered-grid Finite-difference Seismic Wave Modeling on GPU cluster

有限差分实现简单、速度快,作为地震波场模拟一种有效数值方法,被广泛用于正演计算密集的波形反演和逆时偏移中.三维地震波正演模拟计算量大,一直以来制约着三维叠前逆时偏移和反演的工业化应用,GPU通用计算技术的产生及其内在的数据并行性有望改变这一现状.本文通过分析三维交错网格有限差分方法在GPU上的实施,利用片内共享存储器实现了三维地震波数值模拟的高效算法,取得了较单核CPU快79x～108x的加速比;通过区域分解技术将单GPU上不能计算的地质体模型沿Z轴方向进行粗粒度分解,采用消息传递接口交换边界数据,运用MPI+CUDA的方式实现了大尺度三维地震波场模拟,并着重分析了影响GPU并行计算效率的一些关键因素.大尺度三维地震波场模拟的加速实现,为促进叠前逆时偏移和波形反演技术的工业化转化提供了可能,因此具有重要的研究意义.

As an efficient numerical method for seismic modeling, finite difference has been widely used in computation-intensive waveform inversion and reverse-time migration. Although simple and fast, it is still hard to alleviate giant computation cost in three-dimensional seismic modeling that greatly restricts the industrial application of pre-stack reverse-time migration and inversion. The GPU general-purpose computation technology that is well known for its inherent data parallelism is expected to break through this limitation and makes large-scale three- dimensional reverse-time migration and inversion possible in application. In this paper, we present a method to configure the discrete grids on GPU thread blocks that optimally maps the global memory address on global video memory into inner shared memory to achieve maximum efficiency. The tests applied in different scale models show that we can achieve 79x- 108x speedup ratio when compared to traditional single-core CPU. By virtue of domain decomposition, we use coarse-grained algorithm to partition the large-scale geological model along the Z-axis and compute them using hybrid MPI and CUDA codes to realize the large-scale seismic modeling which can hardly be implemented on single GPU. In the experiments, we also analyze some key factors that affect the parallel efficiency of GPU cluster. The methodology we present and the problems we discuss can be used as a basis to promote the industrial application of pre-stack reverse-time migration and inversion that depend heavily on large-scale three- dimensional seismic modeling.