基于NTK理论和改进时间因果的物理信息神经网络加速收敛算法

ACCELERATING CONVERGENCE ALGORITHM FOR PHYSICS-INFORMED NEURAL NETWORKS BASED ON NTK THEORY AND MODIFIED CAUSALITY

物理信息神经网络(physics-informed neural networks,PINNs)是一类将先验物理知识嵌入神经网络的方法,目前已经成为求解偏微分方程领域的研究热点.尽管PINNs在数值模拟方面展现出巨大的应用前景,但它仍然面临收敛缓慢的挑战.文章从神经正切核(neural tangent kernel,NTK)理论出发,通过对单隐藏层神经网络模型进行分析,推出PINNs的神经正切核矩阵具体表达式,并以此进一步分析PINNs收敛速度的影响因素,给出PINNs快速收敛的两个必要条件.应用神经正切核理论分析PINNs领域的3种相关算法(时间因果算法、傅里叶特征嵌入、学习率退火)的加速收敛效果,结果表明这3种算法均不能满足PINNs加速收敛的所有必要条件.文章提出一种动态傅里叶特征嵌入时间因果算法(dynamic Fourier feature embedding causality,DFFEC),综合考虑了NTK矩阵特征值平衡和时间顺序收敛对PINNs收敛速度的影响,在AllenCahn,Reaction,Burgers和Advection等4个算例上的数值实验结果表明,所提出的DFFEC算法可以显著提高PINNs的收敛速度.特别是在Allen-Cahn算例上,与时间因果算法相比,所提出的DFFEC算法具有至少50倍的加速收敛效果.

Physics-informed neural networks(PINNs)are a class of neural networks that embed prior physical knowledge into the neural network,and have emerged as a focal area in the study of solving partial differential equations.Despite showing the significant potential in numerical simulation,PINNs still encounter the challenge of slow convergence.Through the lens of neural tangent kernel(NTK)theory,this paper conducts an analysis on single-hiddenlayer neural network models,derives the specific form of the NTK matrix for PINNs,and further analyzes the factors affecting the convergence rate of PINNs,proposing two necessary conditions for high convergence rate.Applying the NTK theory,analysis of three algorithms in the PINNs domain including causality,Fourier feature embedding and learning rate annealing indicates that none of them satisfies all the necessary conditions for high convergence rate.This paper proposes dynamic Fourier feature embedding causality(DFFEC)method which takes both the impact of NTK matrix eigenvalue balance and chronological convergence on the convergence speed into account.The numerical experiments on four benchmark problems including Allen-Cahn,Reaction,Burgers and Advection,illustrate that the proposed DFFEC method can remarkably improve the convergence rate of PINNs.Especially,in the Allen-Cahn case,the proposed DFFEC method achieves an acceleration effect of at least 50 times compared to the causality algorithm.