We study how to use the SR1 update to realize minimization methods for problems where the storage is critical. We give an update formula which generates matrices using information from the last m iterations. The numer...We study how to use the SR1 update to realize minimization methods for problems where the storage is critical. We give an update formula which generates matrices using information from the last m iterations. The numerical tests show that the method is efficent.展开更多
The degree of numerical linear independence is proposed and discussed. Based on this linear independence theory, a modified limited memory BFGS method is deve loped. Similar to the standard limited memory method, thi...The degree of numerical linear independence is proposed and discussed. Based on this linear independence theory, a modified limited memory BFGS method is deve loped. Similar to the standard limited memory method, this new method determines the new update by applying the updating formula m times to an initial positive diagonal matrix using the m previous pairs of the change in iteration and gradient. Besides the most recent pair of the change, which guarantees the quadratic termination, the choice of the other ( m -1) pairs of the change in the new method is dependent on the degree of numerical linear independence of previous search directions. In addition, the numerical linear independence theory is further discussed and the computation of the degree of linear independence is simplified. Theoretical and numerical results show that this new modified method improves efficiently the standard limited memory method.展开更多
In this paper, a new limited memory quasi-Newton method is proposed and developed for solving large-scale linearly equality-constrained nonlinear programming problems. In every iteration, a linear equation subproblem ...In this paper, a new limited memory quasi-Newton method is proposed and developed for solving large-scale linearly equality-constrained nonlinear programming problems. In every iteration, a linear equation subproblem is solved by using the scaled conjugate gradient method. A truncated solution of the subproblem is determined so that computation is decreased. The technique of limited memory is used to update the approximated inverse Hessian matrix of the Lagrangian function. Hence, the new method is able to handle large dense problems. The convergence of the method is analyzed and numerical results are reported.展开更多
In full waveform inversion （FWI）, Hessian information of the misfit function is of vital importance for accelerating the convergence of the inversion; however, it usually is not feasible to directly calculate the He...In full waveform inversion （FWI）, Hessian information of the misfit function is of vital importance for accelerating the convergence of the inversion; however, it usually is not feasible to directly calculate the Hessian matrix and its inverse. Although the limited memory Broyden-Fletcher-Goldfarb-Shanno （L-BFGS） or Hessian-free inexact Newton （HFN） methods are able to use approximate Hessian information, the information they collect is limited. The two methods can be interlaced because they are able to provide Hessian information for each other; however, the performance of the hybrid iterative method is dependent on the effective switch between the two methods. We have designed a new scheme to realize the dynamic switch between the two methods based on the decrease ratio （DR） of the misfit function （objective function）, and we propose a modified hybrid iterative optimization method. In the new scheme, we compare the DR of the two methods for a given computational cost, and choose the method with a faster DR. Using these steps, the modified method always implements the most efficient method. The results of Marmousi and overthrust model testings indicate that the convergence with our modified method is significantly faster than that in the L-BFGS method with no loss of inversion quality. Moreover, our modified outperforms the enriched method by a little speedup of the convergence. It also exhibits better efficiency than the HFN method.展开更多
In this paper, we propose an algorithm for solving nonlinear monotone equations by combining the limited memory BFGS method （L-BFGS） with a projection method. We show that the method is globally convergent if the eq...In this paper, we propose an algorithm for solving nonlinear monotone equations by combining the limited memory BFGS method （L-BFGS） with a projection method. We show that the method is globally convergent if the equation involves a Lipschitz continuous monotone function. We also present some preliminary numerical results.展开更多
In this paper, a modified limited memory BFGS method for solving large-scale unconstrained optimization problems is proposed. A remarkable feature of the proposed method is that it possesses global convergence propert...In this paper, a modified limited memory BFGS method for solving large-scale unconstrained optimization problems is proposed. A remarkable feature of the proposed method is that it possesses global convergence property without convexity assumption on the objective function. Under some suitable conditions, the global convergence of the proposed method is proved. Some numerical results are reported which illustrate that the proposed method is efficient.展开更多
Predicting the external flow field with limited data or limited measurements has attracted long-time interests of researchers in many industrial applications.Physics informed neural network(PINN)provides a seamless fr...Predicting the external flow field with limited data or limited measurements has attracted long-time interests of researchers in many industrial applications.Physics informed neural network(PINN)provides a seamless framework for combining the measured data with the deep neural network,making the neural network capable of executing certain physical constraints.Unlike the data-driven model to learn the end-to-end mapping between the sensor data and high-dimensional flow field,PINN need no prior high-dimensional field as the training dataset and can construct the mapping from sensor data to high dimensional flow field directly.However,the extrapolation of the flow field in the temporal direction is limited due to the lack of training data.Therefore,we apply the long short-term memory(LSTM)network and physics-informed neural network(PINN)to predict the flow field and hydrodynamic force in the future temporal domain with limited data measured in the spatial domain.The physical constraints(conservation laws of fluid flow,e.g.,Navier-Stokes equations)are embedded into the loss function to enforce the trained neural network to capture some latent physical relation between the output fluid parameters and input tempo-spatial parameters.The sparsely measured points in this work are obtained from computational fluid dynamics(CFD)solver based on the local radial basis function(RBF)method.Different numbers of spatial measured points(4–35)downstream the cylinder are trained with/without the prior knowledge of Reynolds number to validate the availability and accuracy of the proposed approach.More practical applications of flow field prediction can compute the drag and lift force along with the cylinder,while different geometry shapes are taken into account.By comparing the flow field reconstruction and force prediction with CFD results,the proposed approach produces a comparable level of accuracy while significantly fewer data in the spatial domain is needed.The numerical results demonstrate that the proposed approach with a specific deep neural network configuration is of great potential for emerging cases where the measured data are often limited.展开更多
Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularl...Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.展开更多
文摘We study how to use the SR1 update to realize minimization methods for problems where the storage is critical. We give an update formula which generates matrices using information from the last m iterations. The numerical tests show that the method is efficent.
文摘The degree of numerical linear independence is proposed and discussed. Based on this linear independence theory, a modified limited memory BFGS method is deve loped. Similar to the standard limited memory method, this new method determines the new update by applying the updating formula m times to an initial positive diagonal matrix using the m previous pairs of the change in iteration and gradient. Besides the most recent pair of the change, which guarantees the quadratic termination, the choice of the other ( m -1) pairs of the change in the new method is dependent on the degree of numerical linear independence of previous search directions. In addition, the numerical linear independence theory is further discussed and the computation of the degree of linear independence is simplified. Theoretical and numerical results show that this new modified method improves efficiently the standard limited memory method.
基金This research is supported by the National Natural Science Foundation of China, LSEC of CAS in Beijingand Natural Science Foun
文摘In this paper, a new limited memory quasi-Newton method is proposed and developed for solving large-scale linearly equality-constrained nonlinear programming problems. In every iteration, a linear equation subproblem is solved by using the scaled conjugate gradient method. A truncated solution of the subproblem is determined so that computation is decreased. The technique of limited memory is used to update the approximated inverse Hessian matrix of the Lagrangian function. Hence, the new method is able to handle large dense problems. The convergence of the method is analyzed and numerical results are reported.
基金financially supported by the National Important and Special Project on Science and Technology(2011ZX05005-005-007HZ)the National Natural Science Foundation of China(No.41274116)
文摘In full waveform inversion （FWI）, Hessian information of the misfit function is of vital importance for accelerating the convergence of the inversion; however, it usually is not feasible to directly calculate the Hessian matrix and its inverse. Although the limited memory Broyden-Fletcher-Goldfarb-Shanno （L-BFGS） or Hessian-free inexact Newton （HFN） methods are able to use approximate Hessian information, the information they collect is limited. The two methods can be interlaced because they are able to provide Hessian information for each other; however, the performance of the hybrid iterative method is dependent on the effective switch between the two methods. We have designed a new scheme to realize the dynamic switch between the two methods based on the decrease ratio （DR） of the misfit function （objective function）, and we propose a modified hybrid iterative optimization method. In the new scheme, we compare the DR of the two methods for a given computational cost, and choose the method with a faster DR. Using these steps, the modified method always implements the most efficient method. The results of Marmousi and overthrust model testings indicate that the convergence with our modified method is significantly faster than that in the L-BFGS method with no loss of inversion quality. Moreover, our modified outperforms the enriched method by a little speedup of the convergence. It also exhibits better efficiency than the HFN method.
基金Support by NSF of China grant 10471036a 973 project
文摘In this paper, we propose an algorithm for solving nonlinear monotone equations by combining the limited memory BFGS method （L-BFGS） with a projection method. We show that the method is globally convergent if the equation involves a Lipschitz continuous monotone function. We also present some preliminary numerical results.
基金Supported by National Natural Science Foundation of China(Grant11001075,11161003)Post-doctoral Foundation of China grant 20090461094the Natural Science Foundation of Henan Province Eduction Department grant 2010B110004
文摘In this paper, a modified limited memory BFGS method for solving large-scale unconstrained optimization problems is proposed. A remarkable feature of the proposed method is that it possesses global convergence property without convexity assumption on the objective function. Under some suitable conditions, the global convergence of the proposed method is proved. Some numerical results are reported which illustrate that the proposed method is efficient.
基金supported by the National Natural Science Foundation of China(Grant Nos.52206053,52130603)。
文摘Predicting the external flow field with limited data or limited measurements has attracted long-time interests of researchers in many industrial applications.Physics informed neural network(PINN)provides a seamless framework for combining the measured data with the deep neural network,making the neural network capable of executing certain physical constraints.Unlike the data-driven model to learn the end-to-end mapping between the sensor data and high-dimensional flow field,PINN need no prior high-dimensional field as the training dataset and can construct the mapping from sensor data to high dimensional flow field directly.However,the extrapolation of the flow field in the temporal direction is limited due to the lack of training data.Therefore,we apply the long short-term memory(LSTM)network and physics-informed neural network(PINN)to predict the flow field and hydrodynamic force in the future temporal domain with limited data measured in the spatial domain.The physical constraints(conservation laws of fluid flow,e.g.,Navier-Stokes equations)are embedded into the loss function to enforce the trained neural network to capture some latent physical relation between the output fluid parameters and input tempo-spatial parameters.The sparsely measured points in this work are obtained from computational fluid dynamics(CFD)solver based on the local radial basis function(RBF)method.Different numbers of spatial measured points(4–35)downstream the cylinder are trained with/without the prior knowledge of Reynolds number to validate the availability and accuracy of the proposed approach.More practical applications of flow field prediction can compute the drag and lift force along with the cylinder,while different geometry shapes are taken into account.By comparing the flow field reconstruction and force prediction with CFD results,the proposed approach produces a comparable level of accuracy while significantly fewer data in the spatial domain is needed.The numerical results demonstrate that the proposed approach with a specific deep neural network configuration is of great potential for emerging cases where the measured data are often limited.
文摘Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.
文摘航空电磁测量记录中,不仅感生电动势测最数据有观测误差,而且高度计测量数据也有误差,直接进行常规反演往往导致反演结果不可靠,研究飞行高度数据有误差下的反演算法具有实际意义.本文以层状模型的固定翼时间域航空电磁多分量理论响应数据为例,提出了两种针对飞行高度计记录数据有误差时的正则化反演算法,一个是自适应正则化反演方法,另一个是约束优化反演方法,结合光滑化模型约束方式,将飞行高度作为一个待反演参数与电阻率参数一并反演,以获得更可靠的解释结果.第一种算法侧重于已知的地电信息相对较少的一般情况下的反演,只要给定初始飞行高度值和初始均匀半空间模型的电阻率值,即可稳定地同时重构地下介质电阻率和飞行高度.反演中正则因子由自适应的方式获得,并用奇异值分解法解反演方程.而第二种算法则用于先验信息较多的特殊场合,可事先设定反演模型参数及飞行高度参数的上下限范围,并通过有限内存拟牛顿约束优化方法搜索可行域里的最优解.用多层介质模型的理论响应数据加入不同水平噪声后进行反演试算,对使用不同飞行高度初值和不同约束参数时的反演结果作对比分析.结果表明,无论飞行高度值偏高或偏低,两种算法均能稳定有效地重构地下介质电导率分布和飞行高度值,但飞行高度初值不准会降低反演的收敛速度;文章的一个算例显示,在飞行高度初值偏低15 m下,第一种算法在第10次迭代后的解释高度与真值的误差小于0.3 m,第二种算法在参数约束下,第6次迭代以后的各次迭代的飞行高度值在119.4～121 m之间,其平均值与真值的误差不足0.2 m.
