MetaPINNs:Predicting soliton and rogue wave of nonlinear PDEs via the improved physics-informed neural networks based on meta-learned optimization

Efficiently solving partial differential equations(PDEs)is a long-standing challenge in mathematics and physics research.In recent years,the rapid development of artificial intelligence technology has brought deep learning-based methods to the forefront of research on numerical methods for partial differential equations.Among them,physics-informed neural networks(PINNs)are a new class of deep learning methods that show great potential in solving PDEs and predicting complex physical phenomena.In the field of nonlinear science,solitary waves and rogue waves have been important research topics.In this paper,we propose an improved PINN that enhances the physical constraints of the neural network model by adding gradient information constraints.In addition,we employ meta-learning optimization to speed up the training process.We apply the improved PINNs to the numerical simulation and prediction of solitary and rogue waves.We evaluate the accuracy of the prediction results by error analysis.The experimental results show that the improved PINNs can make more accurate predictions in less time than that of the original PINNs.

TCAS-PINN:Physics-informed neural networks with a novel temporal causality-based adaptive sampling method

Physics-informed neural networks(PINNs)have become an attractive machine learning framework for obtaining solutions to partial differential equations(PDEs).PINNs embed initial,boundary,and PDE constraints into the loss function.The performance of PINNs is generally affected by both training and sampling.Specifically,training methods focus on how to overcome the training difficulties caused by the special PDE residual loss of PINNs,and sampling methods are concerned with the location and distribution of the sampling points upon which evaluations of PDE residual loss are accomplished.However,a common problem among these original PINNs is that they omit special temporal information utilization during the training or sampling stages when dealing with an important PDE category,namely,time-dependent PDEs,where temporal information plays a key role in the algorithms used.There is one method,called Causal PINN,that considers temporal causality at the training level but not special temporal utilization at the sampling level.Incorporating temporal knowledge into sampling remains to be studied.To fill this gap,we propose a novel temporal causality-based adaptive sampling method that dynamically determines the sampling ratio according to both PDE residual and temporal causality.By designing a sampling ratio determined by both residual loss and temporal causality to control the number and location of sampled points in each temporal sub-domain,we provide a practical solution by incorporating temporal information into sampling.Numerical experiments of several nonlinear time-dependent PDEs,including the Cahn–Hilliard,Korteweg–de Vries,Allen–Cahn and wave equations,show that our proposed sampling method can improve the performance.We demonstrate that using such a relatively simple sampling method can improve prediction performance by up to two orders of magnitude compared with the results from other methods,especially when points are limited.

Physics-informed neural network approach for heat generation rate estimation of lithium-ion battery under various driving conditions

Accurate insight into the heat generation rate(HGR) of lithium-ion batteries(LIBs) is one of key issues for battery management systems to formulate thermal safety warning strategies in advance.For this reason,this paper proposes a novel physics-informed neural network(PINN) approach for HGR estimation of LIBs under various driving conditions.Specifically,a single particle model with thermodynamics(SPMT) is first constructed for extracting the critical physical knowledge related with battery HGR.Subsequently,the surface concentrations of positive and negative electrodes in battery SPMT model are integrated into the bidirectional long short-term memory(BiLSTM) networks as physical information.And combined with other feature variables,a novel PINN approach to achieve HGR estimation of LIBs with higher accuracy is constituted.Additionally,some critical hyperparameters of BiLSTM used in PINN approach are determined through Bayesian optimization algorithm(BOA) and the results of BOA-based BiLSTM are compared with other traditional BiLSTM/LSTM networks.Eventually,combined with the HGR data generated from the validated virtual battery,it is proved that the proposed approach can well predict the battery HGR under the dynamic stress test(DST) and worldwide light vehicles test procedure(WLTP),the mean absolute error under DST is 0.542 kW/m^(3),and the root mean square error under WLTP is1.428 kW/m^(3)at 25℃.Lastly,the investigation results of this paper also show a new perspective in the application of the PINN approach in battery HGR estimation.

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

We consider solving the forward and inverse partial differential equations(PDEs)which have sharp solutions with physics-informed neural networks(PINNs)in this work.In particular,to better capture the sharpness of the solution,we propose the adaptive sampling methods(ASMs)based on the residual and the gradient of the solution.We first present a residual only-based ASM denoted by ASMⅠ.In this approach,we first train the neural network using a small number of residual points and divide the computational domain into a certain number of sub-domains,then we add new residual points in the sub-domain which has the largest mean absolute value of the residual,and those points which have the largest absolute values of the residual in this sub-domain as new residual points.We further develop a second type of ASM(denoted by ASMⅡ)based on both the residual and the gradient of the solution due to the fact that only the residual may not be able to efficiently capture the sharpness of the solution.The procedure of ASMⅡis almost the same as that of ASMⅠ,and we add new residual points which have not only large residuals but also large gradients.To demonstrate the effectiveness of the present methods,we use both ASMⅠand ASMⅡto solve a number of PDEs,including the Burger equation,the compressible Euler equation,the Poisson equation over an Lshape domain as well as the high-dimensional Poisson equation.It has been shown from the numerical results that the sharp solutions can be well approximated by using either ASMⅠor ASMⅡ,and both methods deliver much more accurate solutions than the original PINNs with the same number of residual points.Moreover,the ASMⅡalgorithm has better performance in terms of accuracy,efficiency,and stability compared with the ASMⅠalgorithm.This means that the gradient of the solution improves the stability and efficiency of the adaptive sampling procedure as well as the accuracy of the solution.Furthermore,we also employ the similar adaptive sampling technique for the data points of boundary conditions(BCs)if the sharpness of the solution is near the boundary.The result of the L-shape Poisson problem indicates that the present method can significantly improve the efficiency,stability,and accuracy.

Effective data sampling strategies and boundary condition constraints of physics-informed neural networks for identifying material properties in solid mechanics

Material identification is critical for understanding the relationship between mechanical properties and the associated mechanical functions.However,material identification is a challenging task,especially when the characteristic of the material is highly nonlinear in nature,as is common in biological tissue.In this work,we identify unknown material properties in continuum solid mechanics via physics-informed neural networks(PINNs).To improve the accuracy and efficiency of PINNs,we develop efficient strategies to nonuniformly sample observational data.We also investigate different approaches to enforce Dirichlet-type boundary conditions(BCs)as soft or hard constraints.Finally,we apply the proposed methods to a diverse set of time-dependent and time-independent solid mechanic examples that span linear elastic and hyperelastic material space.The estimated material parameters achieve relative errors of less than 1%.As such,this work is relevant to diverse applications,including optimizing structural integrity and developing novel materials.

Meshfree-based physics-informed neural networks for the unsteady Oseen equations

We propose the meshfree-based physics-informed neural networks for solving the unsteady Oseen equations.Firstly,based on the ideas of meshfree and small sample learning,we only randomly select a small number of spatiotemporal points to train the neural network instead of forming a mesh.Specifically,we optimize the neural network by minimizing the loss function to satisfy the differential operators,initial condition and boundary condition.Then,we prove the convergence of the loss function and the convergence of the neural network.In addition,the feasibility and effectiveness of the method are verified by the results of numerical experiments,and the theoretical derivation is verified by the relative error between the neural network solution and the analytical solution.

An artificial viscosity augmented physics-informed neural network for incompressible flow

Physics-informed neural networks(PINNs)are proved methods that are effective in solving some strongly nonlinear partial differential equations(PDEs),e.g.,Navier-Stokes equations,with a small amount of boundary or interior data.However,the feasibility of applying PINNs to the flow at moderate or high Reynolds numbers has rarely been reported.The present paper proposes an artificial viscosity(AV)-based PINN for solving the forward and inverse flow problems.Specifically,the AV used in PINNs is inspired by the entropy viscosity method developed in conventional computational fluid dynamics(CFD)to stabilize the simulation of flow at high Reynolds numbers.The newly developed PINN is used to solve the forward problem of the two-dimensional steady cavity flow at Re=1000 and the inverse problem derived from two-dimensional film boiling.The results show that the AV augmented PINN can solve both problems with good accuracy and substantially reduce the inference errors in the forward problem.

Physics-informed neural network-based petroleum reservoir simulation with sparse data using domain decomposition

Recent advances in deep learning have expanded new possibilities for fluid flow simulation in petroleum reservoirs.However,the predominant approach in existing research is to train neural networks using high-fidelity numerical simulation data.This presents a significant challenge because the sole source of authentic wellbore production data for training is sparse.In response to this challenge,this work introduces a novel architecture called physics-informed neural network based on domain decomposition(PINN-DD),aiming to effectively utilize the sparse production data of wells for reservoir simulation with large-scale systems.To harness the capabilities of physics-informed neural networks(PINNs)in handling small-scale spatial-temporal domain while addressing the challenges of large-scale systems with sparse labeled data,the computational domain is divided into two distinct sub-domains:the well-containing and the well-free sub-domain.Moreover,the two sub-domains and the interface are rigorously constrained by the governing equations,data matching,and boundary conditions.The accuracy of the proposed method is evaluated on two problems,and its performance is compared against state-of-the-art PINNs through numerical analysis as a benchmark.The results demonstrate the superiority of PINN-DD in handling large-scale reservoir simulation with limited data and show its potential to outperform conventional PINNs in such scenarios.

ESR-PINNs:Physics-informed neural networks with expansion-shrinkage resampling selection strategies

Neural network methods have been widely used in many fields of scientific research with the rapid increase of computing power.The physics-informed neural networks(PINNs)have received much attention as a major breakthrough in solving partial differential equations using neural networks.In this paper,a resampling technique based on the expansion-shrinkage point(ESP)selection strategy is developed to dynamically modify the distribution of training points in accordance with the performance of the neural networks.In this new approach both training sites with slight changes in residual values and training points with large residuals are taken into account.In order to make the distribution of training points more uniform,the concept of continuity is further introduced and incorporated.This method successfully addresses the issue that the neural network becomes ill or even crashes due to the extensive alteration of training point distribution.The effectiveness of the improved physics-informed neural networks with expansion-shrinkage resampling is demonstrated through a series of numerical experiments.

Pre-Training Physics-Informed Neural Network with Mixed Sampling and Its Application in High-Dimensional Systems

Recently,the physics-informed neural network shows remarkable ability in the context of solving the low-dimensional nonlinear partial differential equations.However,for some cases of high-dimensional systems,such technique may be time-consuming and inaccurate.In this paper,the authors put forward a pre-training physics-informed neural network with mixed sampling(pPINN)to address these issues.Just based on the initial and boundary conditions,the authors design the pre-training stage to filter out the set of the misfitting points,which is regarded as part of the training points in the next stage.The authors further take the parameters of the neural network in Stage 1 as the initialization in Stage 2.The advantage of the proposed approach is that it takes less time to transfer the valuable information from the first stage to the second one to improve the calculation accuracy,especially for the high-dimensional systems.To verify the performance of the pPINN algorithm,the authors first focus on the growing-and-decaying mode of line rogue wave in the Davey-Stewartson I equation.Another case is the accelerated motion of lump in the inhomogeneous Kadomtsev-Petviashvili equation,which admits a more complex evolution than the uniform equation.The exact solution provides a perfect sample for data experiments,and can also be used as a reference frame to identify the performance of the algorithm.The experiments confirm that the pPINN algorithm can improve the prediction accuracy and training efficiency well,and reduce the training time to a large extent for simulating nonlinear waves of high-dimensional equations.

Parallel Physics-Informed Neural Networks Method with Regularization Strategies for the Forward-Inverse Problems of the Variable Coefficient Modified KdV Equation

This paper mainly introduces the parallel physics-informed neural networks(PPINNs)method with regularization strategies to solve the data-driven forward-inverse problems of the variable coefficient modified Korteweg-de Vries(VC-MKdV)equation.For the forward problem of the VC-MKdV equation,the authors use the traditional PINN method to obtain satisfactory data-driven soliton solutions and provide a detailed analysis of the impact of network width and depth on solving accuracy and speed.Furthermore,the author finds that the traditional PINN method outperforms the one with locally adaptive activation functions in solving the data-driven forward problems of the VC-MKdV equation.As for the data-driven inverse problem of the VC-MKdV equation,the author introduces a parallel neural networks to separately train the solution function and coefficient function,successfully addressing the function discovery problem of the VC-MKdV equation.To further enhance the network’s generalization ability and noise robustness,the author incorporates two regularization strategies into the PPINNs.An amount of numerical experimental data in this paper demonstrates that the PPINNs method can effectively address the function discovery problem of the VC-MKdV equation,and the inclusion of appropriate regularization strategies in the PPINNs can improves its performance.

A hybrid physics-informed data-driven neural network for CO_(2) storage in depleted shale reservoirs

To reduce CO_(2) emissions in response to global climate change,shale reservoirs could be ideal candidates for long-term carbon geo-sequestration involving multi-scale transport processes.However,most current CO_(2) sequestration models do not adequately consider multiple transport mechanisms.Moreover,the evaluation of CO_(2) storage processes usually involves laborious and time-consuming numerical simulations unsuitable for practical prediction and decision-making.In this paper,an integrated model involving gas diffusion,adsorption,dissolution,slip flow,and Darcy flow is proposed to accurately characterize CO_(2) storage in depleted shale reservoirs,supporting the establishment of a training database.On this basis,a hybrid physics-informed data-driven neural network(HPDNN)is developed as a deep learning surrogate for prediction and inversion.By incorporating multiple sources of scientific knowledge,the HPDNN can be configured with limited simulation resources,significantly accelerating the forward and inversion processes.Furthermore,the HPDNN can more intelligently predict injection performance,precisely perform reservoir parameter inversion,and reasonably evaluate the CO_(2) storage capacity under complicated scenarios.The validation and test results demonstrate that the HPDNN can ensure high accuracy and strong robustness across an extensive applicability range when dealing with field data with multiple noise sources.This study has tremendous potential to replace traditional modeling tools for predicting and making decisions about CO_(2) storage projects in depleted shale reservoirs.

Prediction of Porous Media Fluid Flow with Spatial Heterogeneity Using Criss-Cross Physics-Informed Convolutional Neural Networks

Recent advances in deep neural networks have shed new light on physics,engineering,and scientific computing.Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots.The physicsinformedneural network(PINN)is currently the most general framework,which is more popular due to theconvenience of constructing NNs and excellent generalization ability.The automatic differentiation(AD)-basedPINN model is suitable for the homogeneous scientific problem;however,it is unclear how AD can enforce fluxcontinuity across boundaries between cells of different properties where spatial heterogeneity is represented bygrid cells with different physical properties.In this work,we propose a criss-cross physics-informed convolutionalneural network(CC-PINN)learning architecture,aiming to learn the solution of parametric PDEs with spatialheterogeneity of physical properties.To achieve the seamless enforcement of flux continuity and integration ofphysicalmeaning into CNN,a predefined 2D convolutional layer is proposed to accurately express transmissibilitybetween adjacent cells.The efficacy of the proposedmethodwas evaluated through predictions of several petroleumreservoir problems with spatial heterogeneity and compared against state-of-the-art(PINN)through numericalanalysis as a benchmark,which demonstrated the superiority of the proposed method over the PINN.

Solving nonlinear soliton equations using improved physics-informed neural networks with adaptive mechanisms

Partial differential equations(PDEs)are important tools for scientific research and are widely used in various fields.However,it is usually very difficult to obtain accurate analytical solutions of PDEs,and numerical methods to solve PDEs are often computationally intensive and very time-consuming.In recent years,Physics Informed Neural Networks(PINNs)have been successfully applied to find numerical solutions of PDEs and have shown great potential.All the while,solitary waves have been of great interest to researchers in the field of nonlinear science.In this paper,we perform numerical simulations of solitary wave solutions of several PDEs using improved PINNs.The improved PINNs not only incorporate constraints on the control equations to ensure the interpretability of the prediction results,which is important for physical field simulations,in addition,an adaptive activation function is introduced.By introducing hyperparameters in the activation function to change the slope of the activation function to avoid the disappearance of the gradient,computing time is saved thereby speeding up training.In this paper,the m Kd V equation,the improved Boussinesq equation,the Caudrey–Dodd–Gibbon–Sawada–Kotera equation and the p-g BKP equation are selected for study,and the errors of the simulation results are analyzed to assess the accuracy of the predicted solitary wave solution.The experimental results show that the improved PINNs are significantly better than the traditional PINNs with shorter training time but more accurate prediction results.The improved PINNs improve the training speed by more than 1.5 times compared with the traditional PINNs,while maintaining the prediction error less than 10~(-2)in this order of magnitude.

Application of physics-informed neural network in the analysis of hydrodynamic lubrication

The last decade has witnessed a surge of interest in artificial neural network in many different areas of scientific research.Despite the rapid expansion in the application of neural networks,few efforts have been carried out to introduce such a powerful tool into lubrication studies.Thus,this work aims to apply the physics-informed neural network(PINN)to the hydrodynamic lubrication analysis.The 2D Reynolds equation is solved.The PINN is a meshless method and does not require big data for network training compared with classical methods.Our results are consistent with those obtained by experiments and the finite element method.Hence,we envision that the PINN method will have great application potential in lubrication and bearing research.

Data-driven fusion and fission solutions in the Hirota–Satsuma–Ito equation via the physics-informed neural networks method

Fusion and fission are two important phenomena that have been experimentally observed in many real physical models.In this paper,we investigate the two phenomena in the(2+1)-dimensional Hirota-Satsuma-Ito equation via the physics-informed neural networks(PINN)method.By choosing suitable physically constrained initial boundary conditions,the data-driven fusion and fission solutions are obtained for the first time.Dynamical behaviors and error analysis of these solutions are investigated via illustratively numerical figures,which show that good results are achieved.It is pointed out that the PINN method adopted here can be effectively used to construct the data-driven fusion and fission solutions for other nonlinear integrable equations.Based on the powerful predictive capability of the PINN method and wide applications of fusion and fission in many physical areas,it is hoped that the data-driven solutions obtained here will be helpful for experts to predict or explain related physical phenomena.

Convergence of Physics-Informed Neural Networks Applied to Linear Second-Order Elliptic Interface Problems

With the remarkable empirical success of neural networks across diverse scientific disciplines,rigorous error and convergence analysis are also being developed and enriched.However,there has been little theoretical work focusing on neural networks in solving interface problems.In this paper,we perform a convergence analysis of physics-informed neural networks(PINNs)for solving second-order elliptic interface problems.Specifically,we consider PINNs with domain decomposition technologies and introduce gradient-enhanced strategies on the interfaces to deal with boundary and interface jump conditions.It is shown that the neural network sequence obtained by minimizing a Lipschitz regularized loss function converges to the unique solution to the interface problem in H2 as the number of samples increases.Numerical experiments are provided to demonstrate our theoretical analysis.

An Adaptive Physics-Informed Neural Network with Two-Stage Learning Strategy to Solve Partial Differential Equations

Physics-Informed Neural Network(PINN)represents a new approach to solve Partial Differential Equations(PDEs).PINNs aim to solve PDEs by integrating governing equations and the initial/boundary conditions(I/BCs)into a loss function.However,the imbalance of the loss function caused by parameter settings usually makes it difficult for PINNs to converge,e.g.because they fall into local optima.In other words,the presence of balanced PDE loss,initial loss and boundary loss may be critical for the convergence.In addition,existing PINNs are not able to reveal the hidden errors caused by non-convergent boundaries and conduction errors caused by the PDE near the boundaries.Overall,these problems have made PINN-based methods of limited use on practical situations.In this paper,we propose a novel physics-informed neural network,i.e.an adaptive physics-informed neural network with a two-stage training process.Our algorithm adds spatio-temporal coefficient and PDE balance parameter to the loss function,and solve PDEs using a two-stage training process:pre-training and formal training.The pre-training step ensures the convergence of boundary loss,whereas the formal training process completes the solution of PDE by balancing various loss functions.In order to verify the performance of our method,we consider the imbalanced heat conduction and Helmholtz equations often appearing in practical situations.The Klein-Gordon equation,which is widely used to compare performance,reveals that our method is able to reduce the hidden errors.Experimental results confirm that our algorithm can effectively and accurately solve models with unbalanced loss function,hidden errors and conduction errors.The codes developed in this manuscript are publicy available at https://github.com/callmedrcom/ATPINN.

Physics-informed deep learning for fringe pattern analysis

Recently,deep learning has yielded transformative success across optics and photonics,especially in optical metrology.Deep neural networks (DNNs) with a fully convolutional architecture (e.g.,U-Net and its derivatives) have been widely implemented in an end-to-end manner to accomplish various optical metrology tasks,such as fringe denoising,phase unwrapping,and fringe analysis.However,the task of training a DNN to accurately identify an image-to-image transform from massive input and output data pairs seems at best naive,as the physical laws governing the image formation or other domain expertise pertaining to the measurement have not yet been fully exploited in current deep learning practice.To this end,we introduce a physics-informed deep learning method for fringe pattern analysis (PI-FPA) to overcome this limit by integrating a lightweight DNN with a learning-enhanced Fourier transform profilometry (Le FTP) module.By parameterizing conventional phase retrieval methods,the Le FTP module embeds the prior knowledge in the network structure and the loss function to directly provide reliable phase results for new types of samples,while circumventing the requirement of collecting a large amount of high-quality data in supervised learning methods.Guided by the initial phase from Le FTP,the phase recovery ability of the lightweight DNN is enhanced to further improve the phase accuracy at a low computational cost compared with existing end-to-end networks.Experimental results demonstrate that PI-FPA enables more accurate and computationally efficient single-shot phase retrieval,exhibiting its excellent generalization to various unseen objects during training.The proposed PI-FPA presents that challenging issues in optical metrology can be potentially overcome through the synergy of physics-priors-based traditional tools and data-driven learning approaches,opening new avenues to achieve fast and accurate single-shot 3D imaging.

Physics-informed neural networks for diffraction tomography

We propose a physics-informed neural network(PINN)as the forward model for tomographic reconstructions of biological samples.We demonstrate that by training this network with the Helmholtz equation as a physical loss,we can predict the scattered field accurately.It will be shown that a pretrained network can be fine-tuned for different samples and used for solving the scattering problem much faster than other numerical solutions.We evaluate our methodology with numerical and experimental results.Our PINNs can be generalized for any forward and inverse scattering problem.