A NATURAL GRADIENT ALGORITHM FOR THE SOLUTION OF LYAPUNOV EQUATIONS BASED ON THE GEODESIC DISTANCE

A new framework based on the curved Riemannian manifold is proposed to calculate the numerical solution of the Lyapunov matrix equation by using a natural gradient descent algorithm and taking the geodesic distance as the objective function. Moreover, a gradient descent algorithm based on the classical Euclidean distance is provided to compare with this natural gradient descent algorithm. Furthermore, the behaviors of two proposed algorithms and the conventional modified conjugate gradient algorithm are compared and demonstrated by two simulation examples. By comparison, it is shown that the convergence speed of the natural gradient descent algorithm is faster than both of the gradient descent algorithm and the conventional modified conjugate gradient algorithm in solving the Lyapunov equation.

Natural gradient-based recursive least-squares algorithm for adaptive blind source separation

This paper focuses on the problem of adaptive blind source separation (BSS). First, a recursive least-squares (RLS) whitening algorithm is proposed. By combining it with a natural gradient-based RLS algorithm for nonlinear principle component analysis (PCA), and using reasonable approximations, a novel RLS algorithm which can achieve BSS without additional pre-whitening of the observed mixtures is obtained. Analyses of the equilibrium points show that both of the RLS whitening algorithm and the natural gradient-based RLS algorithm for BSS have the desired convergence properties. It is also proved that the combined new RLS algorithm for BSS is equivariant and has the property of keeping the separating matrix from becoming singular. Finally, the effectiveness of the proposed algorithm is verified by extensive simulation results.

Variational Bayesian Kalman filter using natural gradient

We propose a technique based on the natural gradient method for variational lower bound maximization for a variational Bayesian Kalman filter.The natural gradient approach is applied to the Kullback-Leibler divergence between the parameterized variational distribution and the posterior density of interest.Using a Gaussian assumption for the parametrized variational distribution,we obtain a closed-form iterative procedure for the Kullback-Leibler divergence minimization,producing estimates of the variational hyper-parameters of state estimation and the associated error covariance.Simulation results in both a Doppler radar tracking scenario and a bearing-only tracking scenario are presented,showing that the proposed natural gradient method outperforms existing methods which are based on other linearization techniques in terms of tracking accuracy.

C^1 natural element method for strain gradient linear elasticity and its application to microstructures

C^1 natural element method （C^1 NEM） is applied to strain gradient linear elasticity, and size effects on mi crostructures are analyzed. The shape functions in C^1 NEM are built upon the natural neighbor interpolation （NNI）, with interpolation realized to nodal function and nodal gradient values, so that the essential boundary conditions （EBCs） can be imposed directly in a Galerkin scheme for partial differential equations （PDEs）. In the present paper, C^1 NEM for strain gradient linear elasticity is constructed, and sev- eral typical examples which have analytical solutions are presented to illustrate the effectiveness of the constructed method. In its application to microstructures, the size effects of bending stiffness and stress concentration factor （SCF） are studied for microspeciem and microgripper, respectively. It is observed that the size effects become rather strong when the width of spring for microgripper, the radius of circular perforation and the long axis of elliptical perforation for microspeciem come close to the material characteristic length scales. For the U-shaped notch, the size effects decline obviously with increasing notch radius, and decline mildly with increasing length of notch.

Solution of Algebraic Lyapunov Equation on Positive-Definite Hermitian Matrices by Using Extended Hamiltonian Algorithm

This communique is opted to study the approximate solution of the Algebraic Lyapunov equation on the manifold of positive-definite Hermitian matrices.We choose the geodesic distance between􀀀AHX􀀀XA and P as the cost function,and put forward the Extended Hamiltonian algorithm(EHA)and Natural gradient algorithm(NGA)for the solution.Finally,several numerical experiments give you an idea about the effectiveness of the proposed algorithms.We also show the comparison between these two algorithms EHA and NGA.Obtained results are provided and analyzed graphically.We also conclude that the extended Hamiltonian algorithm has better convergence speed than the natural gradient algorithm,whereas the trajectory of the solution matrix is optimal in case of Natural gradient algorithm(NGA)as compared to Extended Hamiltonian Algorithm(EHA).The aim of this paper is to show that the Extended Hamiltonian algorithm(EHA)has superior convergence properties as compared to Natural gradient algorithm(NGA).Upto the best of author’s knowledge,no approximate solution of the Algebraic Lyapunov equation on the manifold of positive-definite Hermitian matrices is found so far in the literature.

A SIGNAL-ADAPTIVE ALGORITHM FOR BLIND SEPARATION OF SOURCES WITH MIXED KURTOSIS SIGNS

This paper addresses the problem of Blind Source Separation (BSS) and presents a new BSS algorithm with a Signal-Adaptive Activation (SAA) function (SAA-BSS). By taking the sum of absolute values of the normalized kurtoses as a contrast function, the obtained signal-adaptive activation function automatically satisfies the local stability and robustness conditions. The SAA-BSS exploits the natural gradient learning on the Stiefel manifold, and it is an equivariant algorithm with a moderate computational load. Computer simulations show that the SAA-BSS can perform blind separation of mixed sub-Gaussian and super-Gaussian signals and it works more efficiently than the existing algorithms in convergence speed and robustness against outliers.

Blind source separation with unknown and dynamically changing number of source signals

The contrast function remains to be an open problem in blind source separation （BSS） when the number of source signals is unknown and/or dynamically changed. The paper studies this problem and proves that the mutual information is still the contrast function for BSS if the mixing matrix is of full column rank. The mutual information reaches its minimum at the separation points, where the random outputs of the BSS system are the scaled and permuted source signals, while the others are zero outputs. Using the property that the transpose of the mixing matrix and a matrix composed by m observed signals have the indentical null space with probability one, a practical method, which can detect the unknown number of source signals n, ulteriorly traces the dynamical change of the sources number with a few of data, is proposed. The effectiveness of the proposed theorey and the developed novel algorithm is verified by adaptive BSS simulations with unknown and dynamically changing number of source signals.

Adaptive blind separation of underdetermined mixtures based on sparse component analysis

The independence priori is very often used in the conventional blind source separation （BSS）. Naturally, independent component analysis （ICA） is also employed to perform BSS very often. However, ICA is difficult to use in some challenging cases, such as underdetermined BSS or blind separation of dependent sources. Recently, sparse component analysis （SCA） has attained much attention because it is theoretically available for underdetermined BSS and even for blind dependent source separation sometimes. However, SCA has not been developed very sufficiently. Up to now, there are only few existing algorithms and they are also not perfect as well in practice. For example, although Lewicki-Sejnowski＇s natural gradient for SCA is superior to K-mean clustering, it is just an approximation without rigorously theoretical basis. To overcome these problems, a new natural gradient formula is proposed in this paper. This formula is derived directly from the cost function of SCA through matrix theory. Mathematically, it is more rigorous. In addition, a new and robust adaptive BSS algorithm is developed based on the new natural gradient. Simulations illustrate that this natural gradient formula is more robust and reliable than Lewicki-Sejnowski＇s gradient.