A gradually descent method for discrete global optimization

In this paper, a new method named as the gradually descent method was proposed to solve the discrete global optimization problem. With the aid of an auxiliary function, this method enables to convert the problem of finding one discrete minimizer of the objective function f to that of finding another at each cycle. The auxiliary function can ensure that a point, except a prescribed point, is not its integer stationary point if the value of objective function at the point is greater than the scalar which is chosen properly. This property leads to a better minimizer of f found more easily by some classical local search methods. The computational results show that this algorithm is quite efficient and reliable for solving nonlinear integer programming problems.

An Enhanced Steepest Descent Method for Global Optimization-Based Mesh Smoothing

In order to speed up the global optimization-based mesh smoothing, an enhanced steepest descent method is presented in the paper. Numerical experiment results show that the method performs better than the steepest descent method in the global smoothing. We also presented a physically-based interpretation to explain why the method works better than the steepest descent method.

Ionospheric forecasting model using fuzzy logic-based gradient descent method

Space weather phenomena cause satellite to ground or satellite to aircraft transmission outages over the VHF to L-band frequency range, particularly in the low latitude region. Global Positioning System （GPS） is primarily susceptible to this form of space weather. Faulty GPS signals are attributed to ionospheric error, which is a function of Total Electron Content （TEC）. Importantly, precise forecasts of space weather conditions and appropriate hazard observant cautions required for ionospheric space weather obser- vations are limited. In this paper, a fuzzy logic-based gradient descent method has been proposed to forecast the ionospheric TEC values. In this technique, membership functions have been tuned based on the gradient descent estimated values. The proposed algorithm has been tested with the TEC data of two geomagnetic storms in the low latitude station of KL University, Guntur, India （16.44°N, 80.62°E）. It has been found that the gradient descent method performs well and the predicted TEC values are close to the original TEC measurements.

Designing fuzzy inference system based on improved gradient descent method

The distribution of sampling data influences completeness of rule base so that extrapolating missing rules is very difficult. Based on data mining, a self-learning method is developed for identifying fuzzy model and extrapolating missing rules, by means of confidence measure and the improved gradient descent method. The proposed approach can not only identify fuzzy model, update its parameters and determine optimal output fuzzy sets simultaneously, but also resolve the uncontrollable problem led by the regions that data do not cover. The simulation results show the effectiveness and accuracy of the proposed approach with the classical truck backer-upper control problem verifying.

Three-step relaxed hybrid steepest-descent methods for variational inequalities

The classical variational inequality problem with a Lipschitzian and strongly monotone operator on a nonempty closed convex subset in a real Hilbert space is studied. A new three-step relaxed hybrid steepest-descent method for this class of variational inequalities is introduced. Strong convergence of this method is established under suitable assumptions imposed on the algorithm parameters.

EFFICIENT GRADIENT DESCENT METHOD OFRBF NEURAL ENTWORKS WITHADAPTIVE LEARNING RATE

A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by "award-punish" strategy. Detailed deduction of the algorithm applied to RBF networks is given. Simulation studies show that this algorithm can increase the rate of convergence and improve the performance of the gradient descent method.

Supervised descent method for weld pool boundary extraction during fiber laser welding process

In order to obtain a high-quality weld during the laser welding process, extracting the characteristic parameters of weld pool is an important issue for automated welding. In this paper, the type 304 austenitic stainless steel is welded by a 5 kW high-power fiber laser and a high-speed camera is employed to capture the topside images of weld pools. Then we propose a robust visual-detection approach for the molten pool based on the supervised descent method. It provides an elegant framework for representing the outline of a weld pool and is especially efficient for weld pool detection in the presence of strong uncertainties and disturbances. Finally, welding experimental results verified that the proposed approach can extract the weld pool boundary accurately, which will lay a solid foundation for controlling the weld quality of fiber laser welding process.

Antenna Array Pattern Synthesis via Coordinate Descent Method

This paper presents an array pattern synthesis algorithm for arbitrary arrays based on coordinate descent method (CDM). With this algorithm, the complex element weights are found to minimize a weighted L2 norm of the difference between desired and achieved pattern. Compared with traditional optimization techniques, CDM is easy to implement and efficient to reach the optimum solutions. Main advantage is the flexibility. CDM is suitable for linear and planar array with arbitrary array elements on arbitrary positions. With this method, we can configure arbitrary beam pattern, which gives it the ability to solve variety of beam forming problem, e.g. focused beam, shaped beam, nulls at arbitrary direction and with arbitrary beam width. CDM is applicable for phase-only and amplitude-only arrays as well, and furthermore, it is a suitable method to treat the problem of array with element failures.

A Stochastic Gradient Descent Method for Computational Design of Random Rough Surfaces in Solar Cells

In this work,we develop a stochastic gradient descent method for the computational optimal design of random rough surfaces in thin-film solar cells.We formulate the design problems as random PDE-constrained optimization problems and seek the optimal statistical parameters for the random surfaces.The optimizations at fixed frequency as well as at multiple frequencies and multiple incident angles are investigated.To evaluate the gradient of the objective function,we derive the shape derivatives for the interfaces and apply the adjoint state method to perform the computation.The stochastic gradient descent method evaluates the gradient of the objective function only at a few samples for each iteration,which reduces the computational cost significantly.Various numerical experiments are conducted to illustrate the efficiency of the method and significant increases of the absorptance for the optimal random structures.We also examine the convergence of the stochastic gradient descent algorithm theoretically and prove that the numerical method is convergent under certain assumptions for the random interfaces.

A Gradient Descent Method for Estimating the Markov Chain Choice Model

In this paper,we propose a gradient descent method to estimate the parameters in a Markov chain choice model.Particularly,we derive closed-form formula for the gradient of the log-likelihood function and show the convergence of the algorithm.Numerical experiments verify the efficiency of our approach by comparing with the expectation-maximization algorithm.We show that the similar result can be extended to a more general case that one does not have observation of the no-purchase data.

A GLOBALLY DERIVTIVE-FREE DESCENT METHOD FOR NONLINEAR COMPLEMENTARITY PROBLEMS

Based on a class of functions, which generalize the squared Fischer-Burmeister NCP function and have many desirable properties as the latter function has, we reformulate nonlinear complementarity problem (NCP for short) as an equivalent unconstrained optimization problem, for which we propose a derivative-free de- scent method in monotone case. We show its global convergence under some mild conditions. If F, the function involved in NCP, is Ro-function, the optimization problem has bounded level sets. A local property of the merit function is discussed. Finally, we report some numerical results.

Fractional-order global optimal backpropagation machine trained by an improved fractional-order steepest descent method

We introduce the fractional-order global optimal backpropagation machine,which is trained by an improved fractionalorder steepest descent method(FSDM).This is a fractional-order backpropagation neural network(FBPNN),a state-of-the-art fractional-order branch of the family of backpropagation neural networks(BPNNs),different from the majority of the previous classic first-order BPNNs which are trained by the traditional first-order steepest descent method.The reverse incremental search of the proposed FBPNN is in the negative directions of the approximate fractional-order partial derivatives of the square error.First,the theoretical concept of an FBPNN trained by an improved FSDM is described mathematically.Then,the mathematical proof of fractional-order global optimal convergence,an assumption of the structure,and fractional-order multi-scale global optimization of the FBPNN are analyzed in detail.Finally,we perform three(types of)experiments to compare the performances of an FBPNN and a classic first-order BPNN,i.e.,example function approximation,fractional-order multi-scale global optimization,and comparison of global search and error fitting abilities with real data.The higher optimal search ability of an FBPNN to determine the global optimal solution is the major advantage that makes the FBPNN superior to a classic first-order BPNN.

Numerical partitioning of components for four-modal sedimentary grain-size distribution based on gradient descent method

The gradient descent(GD)method is used to fit the measured data(i.e.,the laser grain-size distribution of the sediments)with a sum of four weighted lognormal functions.The method is calibrated by a series of ideal numerical experiments.The numerical results indicate that the GD method not only is easy to operate but also could effectively optimize the parameters of the fitting function with the error decreasing steadily.The method is applied to numerical partitioning of laser grain-size components of a series of Garzêloess samples and three bottom sedimentary samples of submarine turbidity currents modeled in an open channel laboratory flume.The overall fitting results are satisfactory.As a new approach of data fitting,the GD method could also be adapted to solve other optimization problems.

Convergence of Hybrid Steepest-Descent Methods for Generalized Variational Inequalities

In this paper, we consider the generalized variational inequality GVI（F, g, C）, where F and g are mappings from a Hilbert space into itself and C is the fixed point set of a nonexpansive mapping. We propose two iterative algorithms to find approximate solutions of the GVI（F,g, C）. Strong convergence results are established and applications to constrained generalized pseudo-inverse are included.

CONVERGENCE ANALYSIS OF A LOCALLY ACCELERATED PRECONDITIONED STEEPEST DESCENT METHOD FOR HERMITIAN-DEFINITE GENERALIZED EIGENVALUE PROBLEMS

By extending the classical analysis techniques due to Samokish, Faddeev and Faddee- va, and Longsine and McCormick among others, we prove the convergence of the precon- ditioned steepest descent with implicit deflation （PSD-id） method for solving Hermitian- definite generalized eigenvalue problems. Furthermore, we derive a nonasymptotie estimate of the rate of convergence of the PSD-id method. We show that with a proper choice of the shift, the indefinite shift-and-invert preconditioner is a locally accelerated preconditioner, and is asymptotically optimal which leads to superlinear convergence Numerical examples are presented to verify the theoretical results on the convergence behavior of the PSD- id method for solving ill-conditioned Hermitian-definite generalized eigenvalue problems arising from electronic structure calculations. While rigorous and full-scale convergence proofs of preconditioned block steepest descent methods in practical use still largely eludes us, we believe the theoretical results presented in this paper shed light on an improved understanding of the convergence behavior of these block methods.

A descent method for the Dubins traveling salesman problem with neighborhoods

In this study,we focus mainly on the problem of finding the minimum-length path through a set of circular regions by a fixed-wing unmanned aerial vehicle.Such a problem is referred to as the Dubins traveling salesman problem with neighborhoods(DTSPN).Algorithms developed in the literature for solving DTSPN either are computationally demanding or generate low-quality solutions.To achieve a better trade-off between solution quality and computational cost,an efficient gradient-free descent method is designed.The core idea of the descent method is to decompose DTSPN into a series of subproblems,each of which consists of finding the minimum-length path of a Dubins vehicle from a configuration to another configuration via an intermediate circular region.By analyzing the geometric properties of the subproblems,we use a bisection method to solve the subproblems.As a result,the descent method can efficiently address DTSPN by successively solving a series of subproblems.Finally,several numerical experiments are carried out to demonstrate the descent method in comparison with several existing algorithms.

A modified three–term conjugate gradient method with sufficient descent property

A hybridization of the three–term conjugate gradient method proposed by Zhang et al. and the nonlinear conjugate gradient method proposed by Polak and Ribi`ere, and Polyak is suggested. Based on an eigenvalue analysis, it is shown that search directions of the proposed method satisfy the sufficient descent condition, independent of the line search and the objective function convexity. Global convergence of the method is established under an Armijo–type line search condition. Numerical experiments show practical efficiency of the proposed method.

基于优化快速搜索随机树算法的全局路径规划

为了改善传统快速搜索随机树(RRT)算法在全局路径规划中存在的平滑度差、具有潜在碰撞性等问题,提出了一种双重优化的RRT算法。在传统RRT算法基础上,引入自适应目标偏向策略以缩短采样时间,引入角度约束采样策略以适应车辆极限转角。得到初始路径后,建立二项优化函数(即降低路径曲率和远离障碍物),并将其作为基点进行梯度下降二次优化,生成可供车辆行驶、平滑性良好且碰撞概率低的路径,并进行仿真验证。结果表明:优化RRT算法相比于传统RRT算法、RRT-Connect算法和RRT算法,平均曲率分别降低了38.1%、36.4%和24.7%,曲率均方差分别降低了38.4%、38.4%和27.2%。

A New Descent Nonlinear Conjugate Gradient Method for Unconstrained Optimization

In this paper, a new nonlinear conjugate gradient method is proposed for large-scale unconstrained optimization. The sufficient descent property holds without any line searches. We use some steplength technique which ensures the Zoutendijk condition to be held, this method is proved to be globally convergent. Finally, we improve it, and do further analysis.

A Descent Gradient Method and Its Global Convergence

Y Liu and C Storey(1992)proposed the famous LS conjugate gradient method which has good numerical results.However,the LS method has very weak convergence under the Wolfe-type line search.In this paper,we give a new descent gradient method based on the LS method.It can guarantee the sufficient descent property at each iteration and the global convergence under the strong Wolfe line search.Finally,we also present extensive preliminary numerical experiments to show the efficiency of the proposed method by comparing with the famous PRP^+method.