An Adaptive Spectral Conjugate Gradient Method with Restart Strategy

As a generalization of the two-term conjugate gradient method(CGM),the spectral CGM is one of the effective methods for solving unconstrained optimization.In this paper,we enhance the JJSL conjugate parameter,initially proposed by Jiang et al.(Computational and Applied Mathematics,2021,40:174),through the utilization of a convex combination technique.And this improvement allows for an adaptive search direction by integrating a newly constructed spectral gradient-type restart strategy.Then,we develop a new spectral CGM by employing an inexact line search to determine the step size.With the application of the weak Wolfe line search,we establish the sufficient descent property of the proposed search direction.Moreover,under general assumptions,including the employment of the strong Wolfe line search for step size calculation,we demonstrate the global convergence of our new algorithm.Finally,the given unconstrained optimization test results show that the new algorithm is effective.

Almost Sure Convergence of Proximal Stochastic Accelerated Gradient Methods

Proximal gradient descent and its accelerated version are resultful methods for solving the sum of smooth and non-smooth problems. When the smooth function can be represented as a sum of multiple functions, the stochastic proximal gradient method performs well. However, research on its accelerated version remains unclear. This paper proposes a proximal stochastic accelerated gradient (PSAG) method to address problems involving a combination of smooth and non-smooth components, where the smooth part corresponds to the average of multiple block sums. Simultaneously, most of convergence analyses hold in expectation. To this end, under some mind conditions, we present an almost sure convergence of unbiased gradient estimation in the non-smooth setting. Moreover, we establish that the minimum of the squared gradient mapping norm arbitrarily converges to zero with probability one.

Selecting the optimum location of the corner using gravity gradient method

The conventional gravity gradient method to plot the geologic body location is fuzzy. When the depth is large and the geologic body is small, the Vzz and Vzx derivative errors are also large. We describe that using the status distinguishing factor to optimally determine the comer location is more accurate than the conventional higher-order derivative method. Thus, a better small geologic body and fault resolution is obtained by using the gravity gradient method and trial theoretical model calculation. The actual data is better processed, providing a better basis for prospecting and determination of subsurface geologic structure.

Full waveform inversion with spectral conjugategradient method

Spectral conjugate gradient method is an algorithm obtained by combination of spectral gradient method and conjugate gradient method,which is characterized with global convergence and simplicity of spectral gradient method,and small storage of conjugate gradient method.Besides,the spectral conjugate gradient method was proved that the search direction at each iteration is a descent direction of objective function even without relying on any line search method.Spectral conjugate gradient method is applied to full waveform inversion for numerical tests on Marmousi model.The authors give a comparison on numerical results obtained by steepest descent method,conjugate gradient method and spectral conjugate gradient method,which shows that the spectral conjugate gradient method is superior to the other two methods.

GLOBAL CONVERGENCE RESULTS OF A THREE TERM MEMORY GRADIENT METHOD WITH A NON-MONOTONE LINE SEARCH TECHNIQUE

In this paper, a new class of three term memory gradient method with non-monotone line search technique for unconstrained optimization is presented. Global convergence properties of the new methods are discussed. Combining the quasi-Newton method with the new method, the former is modified to have global convergence property. Numerical results show that the new algorithm is efficient.

Preparation of berberine hydrochloride long-circulating liposomes by ionophore A23187-mediated ZnSO_(4)gradient method

The aim of the study was to prepare berberine hydrochloride long-circulating liposomes and optimize the formulation and process parameters,and investigate the influence of different factors on the encapsulation efficiency.Berberine hydrochloride liposomes were prepared in response to a transmembrane ion gradient that was established by ionophore A23187.Free and liposomal drug were separated by cation exchange resin,and then the amount of intraliposomal berberine hydrochloride was determined by UV spectrophotometry.The optimized encapsulation efficiency of berberine hydrochloride liposomes was 94.3%
2.1%when the drug-to-lipid ratio was 1:20,and the mean diameter was 146.9 nm
3.2 nm.As a result,the ionophore A23187-mediated ZnSO_(4)gradient method was suitable for the preparation of berberine hydrochloride liposomes that we could get the desired encapsulation efficiency and drug loading.

Convergence of Online Gradient Method with Penalty for BP Neural Networks

Online gradient method has been widely used as a learning algorithm for training feedforward neural networks. Penalty is often introduced into the training procedure to improve the generalization performance and to decrease the magnitude of network weights. In this paper, some weight boundedness and deterministic con- vergence theorems are proved for the online gradient method with penalty for BP neural network with a hidden layer, assuming that the training samples are supplied with the network in a fixed order within each epoch. The monotonicity of the error function with penalty is also guaranteed in the training iteration. Simulation results for a 3-bits parity problem are presented to support our theoretical results.

A modified three–term conjugate gradient method with sufficient descent property

A hybridization of the three–term conjugate gradient method proposed by Zhang et al. and the nonlinear conjugate gradient method proposed by Polak and Ribi`ere, and Polyak is suggested. Based on an eigenvalue analysis, it is shown that search directions of the proposed method satisfy the sufficient descent condition, independent of the line search and the objective function convexity. Global convergence of the method is established under an Armijo–type line search condition. Numerical experiments show practical efficiency of the proposed method.

IMPROVED GRADIENT METHOD FOR MONOTONE AND LIPSCHITZ CONTINUOUS MAPPINGS IN BANACH SPACES

Let C be a nonempty closed convex subset of a 2-uniformly convex and uniformly smooth Banach space E and {An}n∈N be a family of monotone and Lipschitz continuos mappings of C into E＊. In this article, we consider the improved gradient method by the hybrid method in mathematical programming [i0] for solving the variational inequality problem for {AN} and prove strong convergence theorems. And we get several results which improve the well-known results in a real 2-uniformly convex and uniformly smooth Banach space and a real Hilbert space.

High-efciency improved symmetric successive over-relaxation preconditioned conjugate gradient method for solving large-scale finite element linear equations

Fast solving large-scale linear equations in the finite element analysis is a classical subject in computational mechanics. It is a key technique in computer aided engineering （CAE） and computer aided manufacturing （CAM）. This paper presents a high-efficiency improved symmetric successive over-relaxation （ISSOR） preconditioned conjugate gradient （PCG） method, which maintains lelism consistent with the original form. Ideally, the by 50% as compared with the original algorithm. the convergence and inherent paralcomputation can It is suitable for be reduced nearly high-performance computing with its inherent basic high-efficiency operations. By comparing with the numerical results, it is shown that the proposed method has the best performance.

One Step Positive Approach for Sheet Metal Forming Simulation Based on Quasi-conjugate-gradient Method

In one step inverse finite element approach, an initial blank shape is normally predicted from the final deformed shape. The final deformed shape needs to be trimmed into a final part after stamping, the trimmed area, therefore, needs to be compensated manually before using one step inverse approach, which causes low efficiency and in consistency with the real situation. To solve this problem, one step positive approach is proposed to simulate the sheet metal stamping process. Firstly the spatial initial solution of one step positive method is preliminarily obtained by using the mapping relationship and area coordinates, then based on the deformation theory the iterative solving is carried out in three-dimensional coordinate system by using quasi-conjugate-gradient method. During iterative process the contact judgment method is introduced to ensure that the nodes on the spatial initial solution are not separated from die surface. The predicted results of sheet metal forming process that include the shape and thickness of the stamped part can be obtained after the iterative solving process. The validity of the proposed approach is verified by comparing the predicted results obtained through the proposed approach with those obtained through the module of one step inverse approach in Autoform and the real stamped part. In one step positive method, the stamped shape of regular sheet can be calculated fast and effectively. During the iterative solution, the quasi-conjugate-gradient method is proposed to take the place of solving system of equations, and it can improve the stability and precision of the algorithm.

Online Gradient Methods with a Punishing Term for Neural Networks

Online gradient methods are widely used for training the weight of neural networks and for other engineering computations. In certain cases, the resulting weight may become very large, causing difficulties in the implementation of the network by electronic circuits. In this paper we introduce a punishing term into the error function of the training procedure to prevent this situation. The corresponding convergence of the iterative training procedure and the boundedness of the weight sequence are proved. A supporting numerical example is also provided.

Global Convergence of a New Restarting Conjugate Gradient Method for Nonlinear Optimizations

Conjugate gradient optimization algorithms depend on the search directions with different choices for the parameters in the search directions. In this note, by combining the nice numerical performance of PR and HS methods with the global convergence property of the class of conjugate gradient methods presented by HU and STOREY(1991), a class of new restarting conjugate gradient methods is presented. Global convergences of the new method with two kinds of common line searches, are proved. Firstly, it is shown that, using reverse modulus of continuity function and forcing function, the new method for solving unconstrained optimization can work for a continously dif ferentiable function with Curry-Altman's step size rule and a bounded level set. Secondly, by using comparing technique, some general convergence properties of the new method with other kind of step size rule are established. Numerical experiments show that the new method is efficient by comparing with FR conjugate gradient method.

A New Nonlinear Conjugate Gradient Method for Unconstrained Optimization Problems

In this paper,an efficient conjugate gradient method is given to solve the general unconstrained optimization problems,which can guarantee the sufficient descent property and the global convergence with the strong Wolfe line search conditions.Numerical results show that the new method is efficient and stationary by comparing with PRP＋ method,so it can be widely used in scientific computation.

Global Convergence of a New Restarting Three Terms Conjugate Gradient Method for Non-linear Optimizations

In this note,by combining the nice numerical performance of PR and HS methods with the global convergence property of FR method,a class of new restarting three terms conjugate gradient methods is presented.Global convergence properties of the new method with two kinds of common line searches are proved.

Subspace Minimization Conjugate Gradient Method Based on Cubic Regularization Model for Unconstrained Optimization

Many methods have been put forward to solve unconstrained optimization problems,among which conjugate gradient method(CG)is very important.With the increasing emergence of large⁃scale problems,the subspace technology has become particularly important and widely used in the field of optimization.In this study,a new CG method was put forward,which combined subspace technology and a cubic regularization model.Besides,a special scaled norm in a cubic regularization model was analyzed.Under certain conditions,some significant characteristics of the search direction were given and the convergence of the algorithm was built.Numerical comparisons show that for the 145 test functions under the CUTEr library,the proposed method is better than two classical CG methods and two new subspaces conjugate gradient methods.

An Efficient Projected Gradient Method for Convex Constrained Monotone Equations with Applications in Compressive Sensing

In this paper, a modified Polak-Ribière-Polyak conjugate gradient projection method is proposed for solving large scale nonlinear convex constrained monotone equations based on the projection method of Solodov and Svaiter. The obtained method has low-complexity property and converges globally. Furthermore, this method has also been extended to solve the sparse signal reconstruction in compressive sensing. Numerical experiments illustrate the efficiency of the given method and show that such non-monotone method is suitable for some large scale problems.

A New Class of Nonlinear Conjugate Gradient Methods with Global Convergence Properties

非线性共轭梯度法由于其迭代简单和储存量小,且搜索方向不需要满足正割条件,在求解大规模无约束优化问题时占据及其重要的地位.提出了一类新的共轭梯度法,其搜索方向是目标函数的下降方向.若假设目标函数连续可微且梯度满足Lipschitz条件,线性搜索满足Wolfe原则,讨论了所设计算法的全局收敛性.

GLOBAL CONVERGENCE OF THE GENERAL THREE TERM CONJUGATE GRADIENT METHODS WITH THE RELAXED STRONG WOLFE LINE SEARCH

The global convergence of the general three term conjugate gradient methods with the relaxed strong Wolfe line search is proved.

IMPROVED PRECONDITIONED CONJUGATE GRADIENT METHOD AND ITS APPLICATION IN F.E.A.FOR ENGINEERING

In this paper two theorems with theoretical and practical significance are given in respect to the preconditioned conjugate gradient method (PCCG). The theorems discuss respectively the qualitative property of the iterative solution and the construction principle of the iterative matrix. The authors put forward a new incompletely LU factorizing technique for non-M-matrix and the method of constructing the iterative matrix. This improved PCCG is used to calculate the ill-conditioned problems and large-scale three-dimensional finite element problems, and simultaneously contrasted with other methods. The abnormal phenomenon is analyzed when PCCG is used to solve the system of ill-conditioned equations, ft is shown that the method proposed in this paper is quite effective in solving the system of large-scale finite element equations and the system of ill-conditioned equations.