A Hybrid Conjugate Gradient Algorithm for Solving Relative Orientation of Big Rotation Angle Stereo Pair

The fast convergence without initial value dependence is the key to solving large angle relative orientation.Therefore,a hybrid conjugate gradient algorithm is proposed in this paper.The concrete process is:①stochastic hill climbing(SHC)algorithm is used to make a random disturbance to the given initial value of the relative orientation element,and the new value to guarantee the optimization direction is generated.②In local optimization,a super-linear convergent conjugate gradient method is used to replace the steepest descent method in relative orientation to improve its convergence rate.③The global convergence condition is that the calculation error is less than the prescribed limit error.The comparison experiment shows that the method proposed in this paper is independent of the initial value,and has higher accuracy and fewer iterations.

The Irregular Weighted Wavelet Frame Conjugate Gradient Algorithm

The dropping off of data during information transmission and the storage device’s damage etc.often leads the sampled data to be non-uniform.The paper, based on the stability theory of irregular wavelet frame and the irregular weighted wavelet frame operator,proposed an irregular weighted wavelet fame conjugate gradient iterative algorithm for the reconstruction of non-uniformly sampling signal. Compared the experiment results with the iterative algorithm of the Ref.[5],the new algorithm has remarkable advantages in approximation error,running time and so on.

TRANSFORM DOMAIN CONJUGATE GRADIENT ALGORITHM FOR ADAPTIVE FILTERING

This paper proposed a new normalized transform domain conjugate gradient algorithm (NT-CGA), which applies the data independent normalized orthogonal transform technique to approximately whiten the input signal and utilises the modified conjugate gradient method to perform sample-by-sample updating of the filter weights more efficiently. Simulation results illustrated that the proposed algorithm has the ability to provide a fast convergence speed and lower steady-error compared to that of traditional least mean square algorithm (LMSA), normalized transform domain least mean square algorithm (NT- LMSA), Quasi-Newton least mean square algorithm (Q-LMSA) and time domain conjugate gradient algorithm (TD-CGA) when the input signal is heavily coloured.

Adaptive Matrix/Vector Gradient Algorithm for Design of IIR Filters and ARMA Models

This work describes a novel adaptive matrix/vector gradient (AMVG) algorithm for design of IIR filters and ARMA signal models. The AMVG algorithm can track to IIR filters and ARMA systems having poles also outside the unit circle. The time reversed filtering procedure was used to treat the unstable conditions. The SVD-based null space solution was used for the initialization of the AMVG algorithm. We demonstrate the feasibility of the method by designing a digital phase shifter, which adapts to complex frequency carriers in the presence of noise. We implement the half-sample delay filter and describe the envelope detector based on the Hilbert transform filter.

An Accelerated Proximal Gradient Algorithm for Hankel TensorCompletion

In this paper,an accelerated proximal gradient algorithm is proposed for Hankel tensor completion problems.In our method,the iterative completion tensors generated by the new algorithm keep Hankel structure based on projection on the Hankel tensor set.Moreover,due to the special properties of Hankel structure,using the fast singular value thresholding operator of the mode-s unfolding of a Hankel tensor can decrease the computational cost.Meanwhile,the convergence of the new algorithm is discussed under some reasonable conditions.Finally,the numerical experiments show the effectiveness of the proposed algorithm.

Gradient Optimizer Algorithm with Hybrid Deep Learning Based Failure Detection and Classification in the Industrial Environment

Failure detection is an essential task in industrial systems for preventing costly downtime and ensuring the seamlessoperation of the system. Current industrial processes are getting smarter with the emergence of Industry 4.0.Specifically, various modernized industrial processes have been equipped with quite a few sensors to collectprocess-based data to find faults arising or prevailing in processes along with monitoring the status of processes.Fault diagnosis of rotating machines serves a main role in the engineering field and industrial production. Dueto the disadvantages of existing fault, diagnosis approaches, which greatly depend on professional experienceand human knowledge, intellectual fault diagnosis based on deep learning (DL) has attracted the researcher’sinterest. DL reaches the desired fault classification and automatic feature learning. Therefore, this article designs a Gradient Optimizer Algorithm with Hybrid Deep Learning-based Failure Detection and Classification (GOAHDLFDC)in the industrial environment. The presented GOAHDL-FDC technique initially applies continuous wavelettransform (CWT) for preprocessing the actual vibrational signals of the rotating machinery. Next, the residualnetwork (ResNet18) model was exploited for the extraction of features from the vibration signals which are thenfed into theHDLmodel for automated fault detection. Finally, theGOA-based hyperparameter tuning is performedtoadjust the parameter valuesof theHDLmodel accurately.The experimental result analysis of the GOAHDL-FD Calgorithm takes place using a series of simulations and the experimentation outcomes highlight the better resultsof the GOAHDL-FDC technique under different aspects.

A Mini-Batch Proximal Stochastic Recursive Gradient Algorithm with Diagonal Barzilai–Borwein Stepsize

Many machine learning problems can be formulated as minimizing the sum of a function and a non-smooth regularization term.Proximal stochastic gradient methods are popular for solving such composite optimization problems.We propose a minibatch proximal stochastic recursive gradient algorithm SRG-DBB,which incorporates the diagonal Barzilai–Borwein(DBB)stepsize strategy to capture the local geometry of the problem.The linear convergence and complexity of SRG-DBB are analyzed for strongly convex functions.We further establish the linear convergence of SRGDBB under the non-strong convexity condition.Moreover,it is proved that SRG-DBB converges sublinearly in the convex case.Numerical experiments on standard data sets indicate that the performance of SRG-DBB is better than or comparable to the proximal stochastic recursive gradient algorithm with best-tuned scalar stepsizes or BB stepsizes.Furthermore,SRG-DBB is superior to some advanced mini-batch proximal stochastic gradient methods.

Fractional Gradient Descent RBFNN for Active Fault-Tolerant Control of Plant Protection UAVs

With the increasing prevalence of high-order systems in engineering applications, these systems often exhibitsignificant disturbances and can be challenging to model accurately. As a result, the active disturbance rejectioncontroller (ADRC) has been widely applied in various fields. However, in controlling plant protection unmannedaerial vehicles (UAVs), which are typically large and subject to significant disturbances, load disturbances andthe possibility of multiple actuator faults during pesticide spraying pose significant challenges. To address theseissues, this paper proposes a novel fault-tolerant control method that combines a radial basis function neuralnetwork (RBFNN) with a second-order ADRC and leverages a fractional gradient descent (FGD) algorithm.We integrate the plant protection UAV model’s uncertain parameters, load disturbance parameters, and actuatorfault parameters and utilize the RBFNN for system parameter identification. The resulting ADRC exhibits loaddisturbance suppression and fault tolerance capabilities, and our proposed active fault-tolerant control law hasLyapunov stability implications. Experimental results obtained using a multi-rotor fault-tolerant test platformdemonstrate that the proposed method outperforms other control strategies regarding load disturbance suppressionand fault-tolerant performance.

Conjugate Gradient Algorithm in the Four-Dimensional Variational Data Assimilation System in GRAPES

Minimization algorithms are singular components in four-dimensional variational data assimilation(4DVar).In this paper,the convergence and application of the conjugate gradient algorithm(CGA),which is based on the Lanczos iterative algorithm and the Hessian matrix derived from tangent linear and adjoint models using a non-hydrostatic framework,are investigated in the 4DVar minimization.First,the influence of the Gram-Schmidt orthogonalization of the Lanczos vector on the convergence of the Lanczos algorithm is studied.The results show that the Lanczos algorithm without orthogonalization fails to converge after the ninth iteration in the 4DVar minimization,while the orthogonalized Lanczos algorithm converges stably.Second,the convergence and computational efficiency of the CGA and quasi-Newton method in batch cycling assimilation experiments are compared on the 4DVar platform of the Global/Regional Assimilation and Prediction System(GRAPES).The CGA is 40%more computationally efficient than the quasi-Newton method,although the equivalent analysis results can be obtained by using either the CGA or the quasi-Newton method.Thus,the CGA based on Lanczos iterations is better for solving the optimization problems in the GRAPES 4DVar system.

Convergence Analysis of Forgetting Gradient Algorithm by Using Martingale Hyperconvergence Theorem

The stochastic gradient (SG) algorithm has less of a computational burden than the least squares algorithms, but it can not track time varying parameters and has a poor convergence rate. In order to improve the tracking properties of the SG algorithm, the forgetting gradient (FG) algorithm is presented, and its convergence is analyzed by using the martingale hyperconvergence theorem. The results show that: (1) for time invariant deterministic systems, the parameter estimates given by the FG algorithm converge consistently to their true values; (2) for stochastic time varying systems, the parameter tracking error is bounded, that is, the parameter tracking error is small when both the parameter change rate and the observation noise are small.

A policy gradient algorithm integrating long and short-term rewards for soft continuum arm control

The soft continuum arm has extensive application in industrial production and human life due to its superior safety and flexibility. Reinforcement learning is a powerful technique for solving soft arm continuous control problems, which can learn an effective control policy with an unknown system model. However, it is often affected by the high sample complexity and requires huge amounts of data to train, which limits its effectiveness in soft arm control. An improved policy gradient method, policy gradient integrating long and short-term rewards denoted as PGLS, is proposed in this paper to overcome this issue. The shortterm rewards provide more dynamic-aware exploration directions for policy learning and improve the exploration efficiency of the algorithm. PGLS can be integrated into current policy gradient algorithms, such as deep deterministic policy gradient(DDPG). The overall control framework is realized and demonstrated in a dynamics simulation environment. Simulation results show that this approach can effectively control the soft arm to reach and track the targets. Compared with DDPG and other model-free reinforcement learning algorithms, the proposed PGLS algorithm has a great improvement in convergence speed and performance. In addition, a fluid-driven soft manipulator is designed and fabricated in this paper, which can verify the proposed PGLS algorithm in real experiments in the future.

ADAPTIVE EXPONENT SMOOTHING GRADIENT ALGORITHM

由于ＬＭＳ算法具有权调节时间延迟和低通滤波的特性，故提出一种新的自适应指数平滑梯度算法。研究表明，当信号是一个高斯平稳过程时，在参数域｛Ω１：α∈（０，１）｝×｛Ω２：β∈（０，∞）｝上，该算法渐进无偏收敛于维纳解。本文给出了算法收敛性能和性能失调的理论分析以及计算公式。计算机模拟的数值结果表明，该算法是有效的。

Several Variants of the Primal-Dual Hybrid Gradient Algorithm with Applications

By reviewing the primal-dual hybrid gradient algorithm(PDHG)pro-posed by He,You and Yuan(SIAM J.Image Sci.,7(4)(2014),pp.2526–2537),in this paper we introduce four improved schemes for solving a class of saddle-point problems.Convergence properties of the proposed algorithms are ensured based on weak assumptions,where none of the objective functions are assumed to be strongly convex but the step-sizes in the primal-dual updates are more flexible than the pre-vious.By making use of variational analysis,the global convergence and sublinear convergence rate in the ergodic/nonergodic sense are established,and the numer-ical efficiency of our algorithms is verified by testing an image deblurring problem compared with several existing algorithms.

A NATURAL GRADIENT ALGORITHM FOR THE SOLUTION OF LYAPUNOV EQUATIONS BASED ON THE GEODESIC DISTANCE

A new framework based on the curved Riemannian manifold is proposed to calculate the numerical solution of the Lyapunov matrix equation by using a natural gradient descent algorithm and taking the geodesic distance as the objective function. Moreover, a gradient descent algorithm based on the classical Euclidean distance is provided to compare with this natural gradient descent algorithm. Furthermore, the behaviors of two proposed algorithms and the conventional modified conjugate gradient algorithm are compared and demonstrated by two simulation examples. By comparison, it is shown that the convergence speed of the natural gradient descent algorithm is faster than both of the gradient descent algorithm and the conventional modified conjugate gradient algorithm in solving the Lyapunov equation.

Improved gradient iterative algorithms for solving Lyapunov matrix equations

In this paper, an improved gradient iterative （GI） algorithm for solving the Lyapunov matrix equations is studied. Convergence of the improved method for any initial value is proved with some conditions. Compared with the GI algorithm, the improved algorithm reduces computational cost and storage. Finally, the algorithm is tested with GI several numerical examples.

Comparison between iterative wavefront control algorithm and direct gradient wavefront control algorithm for adaptive optics system

Among all kinds of wavefront control algorithms in adaptive optics systems, the direct gradient wavefront control algorithm is the most widespread and common method. This control algorithm obtains the actuator voltages directly from wavefront slopes through pre-measuring the relational matrix between deformable mirror actuators and Hartmann wavefront sensor with perfect real-time characteristic and stability. However, with increasing the number of sub-apertures in wavefront sensor and deformable mirror actuators of adaptive optics systems, the matrix operation in direct gradient algorithm takes too much time, which becomes a major factor influencing control effect of adaptive optics systems. In this paper we apply an iterative wavefront control algorithm to high-resolution adaptive optics systems, in which the voltages of each actuator are obtained through iteration arithmetic, which gains great advantage in calculation and storage. For AO system with thousands of actuators, the computational complexity estimate is about O(n2) ～ O(n3) in direct gradient wavefront control algorithm, while the computational complexity estimate in iterative wavefront control algorithm is about O(n) ～(O(n)3/2), in which n is the number of actuators of AO system. And the more the numbers of sub-apertures and deformable mirror actuators, the more significant advantage the iterative wavefront control algorithm exhibits.

Gradient Descent Algorithm for Small UAV Parameter Estimation System

A gradient descent algorithm with adjustable parameter for attitude estimation is developed,aiming at the attitude measurement for small unmanned aerial vehicle(UAV)in real-time flight conditions.The accelerometer and magnetometer are introduced to construct an error equation with the gyros,thus the drifting characteristics of gyroscope can be compensated by solving the error equation utilized by the gradient descent algorithm.Performance of the presented algorithm is evaluated using a self-proposed micro-electro-mechanical system(MEMS)based attitude heading reference system which is mounted on a tri-axis turntable.The on-ground,turntable and flight experiments indicate that the estimation attitude has a good accuracy.Also,the presented system is compared with an open-source flight control system which runs extended Kalman filter(EKF),and the results show that the attitude control system using the gradient descent method can estimate the attitudes for UAV effectively.

Space-borne antenna adaptive anti-jamming method based on gradient-genetic algorithm

A novel space-borne antenna adaptive anti-jamming method based on the genetic algorithm （GA）, which is combined with gradient-like reproduction operators is presented, to search for the best weight for pattern synthesis in radio frequency （RF）. Combined, the GA＇s the capability of the whole searching is, but not limited by selection of the initial parameter, with the gradient algorithm＇s advantage of fast searching. The proposed method requires a smaller sized initial population and lower computational complexity. Therefore, it is flexible to implement this method in the real-time systems. By using the proposed algorithm, the designer can efficiently control both main-lobe shaping and side-lobe level. Simulation results based on the spot survey data show that the algorithm proposed is efficient and feasible.

Gradient Gene Algorithm: a Fast Optimization Method to MST Problem

The extension of Minimum Spanning Tree(MST) problem is an NP hard problem which does not exit a polynomial time algorithm. In this paper, a fast optimization method on MST problem——the Gradient Gene Algorithm is introduced. Compared with other evolutionary algorithms on MST problem, it is more advanced: firstly, very simple and easy to realize; then, efficient and accurate; finally general on other combination optimization problems.

GENERALIZED CONJUGATE-GRADIENT ALGORITHM AND ITS APPLICATIONS TO SEISMIC TRACE INVERSION

１ＩＮＴＲＯＤＵＣＴＩＯＮＣｕｒｅｎｔｌｙ，ｓｅｉｓｍｉｃｔｒａｃｅｉｎｖｅｒｓｉｏｎｈａｓａｌｒｅａｄｙｂｅｅｎａｎｉｍｐｏｒｔａｎｔｗｏｒｋｉｎｓｅｉｓｍｉｃｄａｔａｐｒｏｃｅｓｓｉｎｇｆｏｒｍｅｔｉｃｕｌｏｕｓｏｉｌｇａｓｅｘｐｌｏｒａｔｉ...