Accelerating the discontinuous Galerkin method for seismic wave propagation simulations using multiple GPUs with CUDA and MPI

We have successfully ported an arbitrary highorder discontinuous Galerkin method for solving the threedimensional isotropic elastic wave equation on unstructured tetrahedral meshes to multiple Graphic Processing Units （GPUs） using the Compute Unified Device Architecture （CUDA） of NVIDIA and Message Passing Interface （MPI） and obtained a speedup factor of about 28.3 for the single-precision version of our codes and a speedup factor of about 14.9 for the double-precision version. The GPU used in the comparisons is NVIDIA Tesla C2070 Fermi, and the CPU used is Intel Xeon W5660. To effectively overlap inter-process communication with computation, we separate the elements on each subdomain into inner and outer elements and complete the computation on outer elements and fill the MPI buffer first. While the MPI messages travel across the network, the GPU performs computation on inner elements, and all other calculations that do not use information of outer elements from neighboring subdomains. A significant portion of the speedup also comes from a customized matrix-matrix multiplication kernel, which is used extensively throughout our program. Preliminary performance analysis on our parallel GPU codes shows favorable strong and weak scalabilities.

Comparative analysis of the characteristics of global seismic wave propagation excited by the M_w 9.0 Tohoku earthquake using numerical simulation

Giant earthquakes generate rich signals that can be used to explore the characteristics of the hierarchical structure of the Earth’s interior associated with the eigenfrequencies of the Earth.We employ the spectral element method,incorporated with large-scale parallel computing technology,to investigate the characteristics of global seismic wave propagation excited by the2011 Mw9.0 Tohoku earthquake.The transversely isotropic PREM model is employed as a prototype of our numerical global Earth model.Topographic data and the effect of the oceans are taken into consideration.Wave propagation processes are simulated by solving three-dimensional elastic wave governing equations with the seismic moment tensor obtained from the Global Centroid Moment Tensor Catalog.Three-dimensional visualization of our computing results displays the nature of the global seismic wave propagation.Comparative analysis of our calculations with observations obtained from the Incorporated Research Institutions for Seismology demonstrates the reliability and feasibility of our numerical results.We compare synthetic seismograms with incorporated and unincorporated ocean models.First results show that the oceans have obvious effects on the characteristics of seismic wave propagation.The peak displacement and peak velocity of P waves become relatively small under the effect of the ocean.However,the effect of the ocean on S-waves is complex.The displacement and velocity of S waves decrease rapidly over time using an unincorporated ocean model.Therefore,the effects of the ocean should be incorporated when undertaking quantitative earthquake hazard assessments on coastal areas.In addition,we undertake comparative analysis on the characteristics of the Earth’s oscillation excited by the 2004 Sumatra-Andaman,2008 Wenchuan,and 2011Tohoku earthquakes that incorporate the effect of the Earth’s gravitational potential.A comparison of the amplitude spectra of the numerical records indicates that energy released by the three big earthquakes is different.Our comparative analysis realizes that the computing results can accurately reproduce some eigenfrequencies of the Earth,such as toroidal modes 0T2 to 0T13and spheroidal modes 0S7 to 0S31.These results demonstrate that numerical simulations can be successfully used to investigate the Earth’s oscillations.We propose that numerical simulations can be used as one of the major tools to further reveal how the Earth’s lateral heterogeneities affect the Earth’s oscillations.

A Preconditioned 3-DMulti-Region Fast Multipole Solver for Seismic Wave Propagation in Complex Geometries

The analysis of seismic wave propagation and amplification in complex geological structures requires efficient numerical methods.In this article,following up on recent studies devoted to the formulation,implementation and evaluation of 3-D single-and multi-region elastodynamic fast multipole boundary element methods(FM-BEMs),a simple preconditioning strategy is proposed.Its efficiency is demonstrated on both the single-andmulti-region versions using benchmark examples(scattering of plane waves by canyons and basins).Finally,the preconditioned FM-BEM is applied to the scattering of plane seismic waves in an actual configuration(alpine basin of Grenoble,France),for which the high velocity contrast is seen to significantly affect the overall efficiency of the multi-region FM-BEM.

Spectral-element simulations of elastic wave propagation in exploration and geotechnical applications

We apply the spectral-element method（SEM）,a high-order finite-element method（FEM） to simulate seismic wave propagation in complex media for exploration and geotechnical problems. The SEM accurately treats geometrical complexities through its flexible FEM mesh and accurately interpolates wavefields through high-order Lagrange polynomials. It has been a numerical solver used extensively in earthquake seismology. We demonstrate the applicability of SEM for selected 2D exploration and geotechnical velocity models with an open-source SEM software package SPECFEM2D. The first scenario involves a marine survey for a salt dome with the presence of major internal discontinuities,and the second example simulates seismic wave propagation for an open-pit mine with complex surface topography. Wavefield snapshots,synthetic seismograms,and peak particle velocity maps are presented to illustrate the promising use of SEM for industrial problems.

Studies on seismic waves

The development of seismic wave study in China in the past four years is reviewed. The discussion is divided into several aspects, including seismic wave propagation in laterally homogeneous media, laterally heterogeneous media, anisotropic and porous media, surface wave and seismic wave inversion, and seismic wave study in prospecting and logging problems. Important projects in the current studies on seismic wave is suggested as the development of high efficient numerical methods, and applying them to the studies of excitation and propagation of seismic waves in complex media and strong ground motion, which will form a foundation for refined earthquake hazard analysis and prediction.

Joint 3D traveltime calculation based on fast marching method and wavefront construction

3D traveltime calculation is widely used in seismic exploration technologies such as seismic migration and tomography. The fast marching method （FMM） is useful for calculating 3D traveltime and has proven to be efficient and stable. However, it has low calculation accuracy near the source, which thus gives it low overall accuracy. This paper proposes a joint traveltime calculation method to solve this problem. The method firstly employs the wavefront construction method （WFC）, which has a higher calculation accuracy than FMM in calculating traveltime in the small area near the source, and secondly adopts FMM to calculate traveltime for the remaining grid nodes. Due to the increase in calculation precision of grid nodes near the source, this new algorithm is shown to have good calculation precision while maintaining the high calculation efficiency of FMM, which is employed in most of the computational area. Results are verified using various numerical models.

Seismic-Wave Scattering,Imaging,and Inversion

Ru-Shan Wu has made seminal contributions in many research areas in geophysics,such as seismic-wave propagation,scattering,imaging,and inversion.We highlight some of his research in holography imaging,diffraction tomography,seismic-wave scattering and its applications to studying Earth’s heterogeneity,oneway wave propagation and one-return wave modeling,beamlet and dreamlet applications,strong non-linear full-waveform inversion,and direct envelop inversion.

A Stable Finite Difference Method for the Elastic Wave Equation on Complex Geometries with Free Surfaces

A stable and explicit second order accurate finite difference method for the elastic wave equation in curvilinear coordinates is presented.The discretization of the spatial operators in the method is shown to be self-adjoint for free-surface,Dirichlet and periodic boundary conditions.The fully discrete version of the method conserves a discrete energy to machine precision.