基于Transformer神经网络的瞬变电磁法数值模拟

Transient electromagnetic numerical simulation based on the transformer neural network

【目的】基于深度学习的瞬变电磁正反演方法属于数据驱动型方法,需要大规模的数值模拟结果作为监督数据对神经网络进行训练和评估。传统的时域有限差分法进行瞬变电磁数值模拟需要迭代求解时域麦克斯韦方程,耗时严重,且耗费计算资源庞大,难以满足基于深度学习的瞬变电磁反演对数据的需求。【方法】为此,引入深度学习方法进行瞬变电磁数值模拟。以Transformer神经网络架构为基础,采用编解码结构(encoder-decoder)设计了基于深度学习的瞬变电磁数值模拟神经网络,该网络包含三维网格剖分模块、分块嵌入模块(patch embedding)、编码器模块(Transformer encoder)、解码器模块(linear decoder) 4部分,输入为模型地电参数,输出为该模型对应的回线源中心点瞬变电磁响应。采用带动量的随机梯度下降算法与Adam自适应学习率算法相结合的优化策略,在搭载4张英伟达V100显卡的服务器上进行了200个周期的训练,获得了瞬变电磁法数值模拟深度学习网络。【结果和结论】利用训练后的网络分别对均匀半空间模型、层状模型、板状体模型以及三维体模型4种类型的地电模型回线源瞬变电磁响应进行实时预测,验证集测试结果表明:计算得到的关断时间后的数值模拟结果与采用解析解、线性数字滤波解和时域有限差分法数值模拟结果平均相关误差小于2%;同时,基于神经网络的瞬变电磁数值模拟计算耗时不到1 s,实现了瞬变电磁数值模拟的快速精确计算,为后续基于深度学习的瞬变电磁反演研究奠定了理论基础与数据支撑。

[Objective]The deep learning-based transient electromagnetic(TEM)forward and inverse modeling methods are data-driven,requiring considerable numerical simulation results as supervisory data to train and assess neural networks.The conventional finite-difference time-domain(FDTD)method for TEM numerical simulation necessitates an iterative solution of time-domain Maxwell's equations.Therefore,this method is time-consuming and computationally intensive,failing to meet the data demand of deep learning-based TEM inversion.[Methods]This study introduced deep learning for TEM numerical simulation.Based on the transformer neural network architecture,a neural network for deep learning-based TEM numerical simulation was designed using an encoder-decoder structure.This neural network comprised a 3D gridding module,a patch embedding module,a transformer encoder module,and a linear decoder module.With geoelectric parameters as inputs,this neural network output corresponding TEM responses at the center of the loop source.It was trained for over 200 epochs using the optimization strategy of stochastic gradient descent with momentum combined with adaptive moment estimation(Adam)−an adaptive learning rate algorithm−on a server equipped with four NVIDIA V100 GPUs.[Results and Conclusions]The trained network for deep learning-based TEM numerical simulation was employed to predict the TEM responses of the loop sources of four geoelectric models in real time: a homogeneous halfspace model, a layered model, a plate model, and a 3D volume model. The test results of thevalidation set reveal that the numerical simulation results after the turn-off time derived using the neural network exhibitedlow mean relative errors (MREs) of less than 2% compared to analytical solutions, linear digital filtering solutions,and FDTD numerical simulation results. Meanwhile, the TEM numerical simulation results were obtained within 1 s.Therefore, the proposed neural network exhibited fast and accurate calculations. This study will provide a theoreticalfoundation and data support for research on deep learning-based TEM inversion.