Vehicle Representation and Classification of Surveillance Video Based on Sparse Learning

We cast vehicle recognition as problem of feature representation and classification, and introduce a sparse learning based framework for vehicle recognition and classification in this paper. After objects captured with a GMM background subtraction program, images are labeled with vehicle type for dictionary learning and decompose the images with sparse coding （SC）, a linear SVM trained with the SC feature for vehicle classification. A simple but efficient active learning stategy is adopted by adding the false positive samples into previous training set for dictionary and SVM model retraining. Compared with traditional feature representation and classification realized with SVM, SC method achieves dramatically improvement on classification accuracy and exhibits strong robustness. The work is also validated on real-world surveillance video.

Joint Multi-Domain Channel Estimation Based on Sparse Bayesian Learning for OTFS System

Since orthogonal time-frequency space(OTFS)can effectively handle the problems caused by Doppler effect in high-mobility environment,it has gradually become a promising candidate for modulation scheme in the next generation of mobile communication.However,the inter-Doppler interference(IDI)problem caused by fractional Doppler poses great challenges to channel estimation.To avoid this problem,this paper proposes a joint time and delayDoppler(DD)domain based on sparse Bayesian learning(SBL)channel estimation algorithm.Firstly,we derive the original channel response(OCR)from the time domain channel impulse response(CIR),which can reflect the channel variation during one OTFS symbol.Compare with the traditional channel model,the OCR can avoid the IDI problem.After that,the dimension of OCR is reduced by using the basis expansion model(BEM)and the relationship between the time and DD domain channel model,so that we have turned the underdetermined problem into an overdetermined problem.Finally,in terms of sparsity of channel in delay domain,SBL algorithm is used to estimate the basis coefficients in the BEM without any priori information of channel.The simulation results show the effectiveness and superiority of the proposed channel estimation algorithm.

Vector Approximate Message Passing with Sparse Bayesian Learning for Gaussian Mixture Prior

Compressed sensing(CS)aims for seeking appropriate algorithms to recover a sparse vector from noisy linear observations.Currently,various Bayesian-based algorithms such as sparse Bayesian learning(SBL)and approximate message passing(AMP)based algorithms have been proposed.For SBL,it has accurate performance with robustness while its computational complexity is high due to matrix inversion.For AMP,its performance is guaranteed by the severe restriction of the measurement matrix,which limits its application in solving CS problem.To overcome the drawbacks of the above algorithms,in this paper,we present a low complexity algorithm for the single linear model that incorporates the vector AMP(VAMP)into the SBL structure with expectation maximization(EM).Specifically,we apply the variance auto-tuning into the VAMP to implement the E step in SBL,which decrease the iterations that require to converge compared with VAMP-EM algorithm when using a Gaussian mixture(GM)prior.Simulation results show that the proposed algorithm has better performance with high robustness under various cases of difficult measurement matrices.

A systematic review of structured sparse learning

High-dimensional data arising from diverse scientific research fields and industrial development have led to increased interest in sparse learning due to model parsimony and computational advantage. With the assumption of sparsity, many computational problems can be handled efficiently in practice. Structured sparse learning encodes the structural information of the variables and has been quite successful in numerous research fields. With various types of structures discovered, sorts of structured regularizations have been proposed. These regularizations have greatly improved the efficacy of sparse learning algorithms through the use of specific structural information. In this article, we present a systematic review of structured sparse learning including ideas, formulations, algorithms, and applications. We present these algorithms in the unified framework of minimizing the sum of loss and penalty functions, summarize publicly accessible software implementations, and compare the computational complexity of typical optimization methods to solve structured sparse learning problems. In experiments, we present applications in unsupervised learning, for structured signal recovery and hierarchical image reconstruction, and in supervised learning in the context of a novel graph-guided logistic regression.

Robust object tracking with RGBD-based sparse learning

Robust object tracking has been an important and challenging research area in the field of computer vision for decades. With the increasing popularity of affordable depth sensors, range data is widely used in visual tracking for its ability to provide robustness to varying illumination and occlusions. In this paper, a novel RGBD and sparse learning based tracker is proposed. The range data is integrated into the sparse learning framework in three respects. First, an extra depth view is added to the color image based visual features as an independent view for robust appearance modeling. Then, a special occlusion template set is designed to replenish the existing dictionary for handling various occlusion conditions. Finally, a depth-based occlusion detection method is proposed to efficiently determine an accurate time for the template update. Extensive experiments on both KITTI and Princeton data sets demonstrate that the proposed tracker outperforms the state-of-the-art tracking algorithms, including both sparse learning and RGBD based methods.

DOA estimation based on multi-frequency joint sparse Bayesian learning for passive radar

This paper considers multi-frequency passive radar and develops a multi-frequency joint direction of arrival(DOA)estimation algorithm to improve estimation accuracy and resolution.The developed algorithm exploits the sparsity of targets in the spatial domain.Specifically,we first extract the required frequency channel data and acquire the snapshot data through a series of preprocessing such as clutter suppression,coherent integration,beamforming,and constant false alarm rate(CFAR)detection.Then,based on the framework of sparse Bayesian learning,the target’s DOA is estimated by jointly extracting the multi-frequency data via evidence maximization.Simulation results show that the developed algorithm has better estimation accuracy and resolution than other existing multi-frequency DOA estimation algorithms,especially under the scenarios of low signalto-noise ratio(SNR)and small snapshots.Furthermore,the effectiveness is verified by the field experimental data of a multi-frequency FM-based passive radar.

Multimodality Prediction of Chaotic Time Series with Sparse Hard-Cut EM Learning of the Gaussian Process Mixture Model

The contribution of this work is twofold： （1） a multimodality prediction method of chaotic time series with the Gaussian process mixture （GPM） model is proposed, which employs a divide and conquer strategy. It automatically divides the chaotic time series into multiple modalities with different extrinsic patterns and intrinsic characteristics, and thus can more precisely fit the chaotic time series. （2） An effective sparse hard-cut expec- tation maximization （SHC-EM） learning algorithm for the GPM model is proposed to improve the prediction performance. SHO-EM replaces a large learning sample set with fewer pseudo inputs, accelerating model learning based on these pseudo inputs. Experiments on Lorenz and Chua time series demonstrate that the proposed method yields not only accurate multimodality prediction, but also the prediction confidence interval SHC-EM outperforms the traditional variational 1earning in terms of both prediction accuracy and speed. In addition, SHC-EM is more robust and insusceptible to noise than variational learning.

EARLY CATARACT DETECTION BY DYNAMIC LIGHT SCATTERING WITH SPARSE BAYESIAN LEARNING

Dynamic light scattering(DLS)is a promising technique for early cataract detection and for studying cataractogenesis.A novel probabilistic analysis tool,the sparse Bayesian learning(SBL)algorithm,is described for reconstructing the most-probable size distribution ofα-crystallin and their aggregates in an ocular lens from the DLS data.The performance of the algorithm is evaluated by analyzing simulated correlation data from known distributions and DLS data from the ocular lenses of a fetal calf,a Rhesus monkey,and a man,so as to establish the required efficiency of the SBL algorithm for clinical studies.

Nonlinear industrial process fault diagnosis with latent label consistency and sparse Gaussian feature learning

With the increasing complexity of industrial processes, the high-dimensional industrial data exhibit a strong nonlinearity, bringing considerable challenges to the fault diagnosis of industrial processes. To efficiently extract deep meaningful features that are crucial for fault diagnosis, a sparse Gaussian feature extractor(SGFE) is designed to learn a nonlinear mapping that projects the raw data into the feature space with the fault label dimension. The feature space is described by the one-hot encoding of the fault category label as an orthogonal basis. In this way, the deep sparse Gaussian features related to fault categories can be gradually learned from the raw data by SGFE. In the feature space,the sparse Gaussian(SG) loss function is designed to constrain the distribution of features to multiple sparse multivariate Gaussian distributions. The sparse Gaussian features are linearly separable in the feature space, which is conducive to improving the accuracy of the downstream fault classification task. The feasibility and practical utility of the proposed SGFE are verified by the handwritten digits MNIST benchmark and Tennessee-Eastman(TE) benchmark process,respectively.

基于吉布斯采样的稀疏水声信道估计方法

The estimation of sparse underwater acoustic(UWA)channels can be regarded as an inference problem involving hidden variables within the Bayesian framework.While the classical sparse Bayesian learning(SBL),derived through the expectation maximization(EM)algorithm,has been widely employed for UWA channel estimation,it still differs from the real posterior expectation of channels.In this paper,we propose an approach that combines variational inference(VI)and Markov chain Monte Carlo(MCMC)methods to provide a more accurate posterior estimation.Specifically,the SBL is first re-derived with VI,allowing us to replace the posterior distribution of the hidden variables with a variational distribution.Then,we determine the full conditional probability distribution for each variable in the variational distribution and then iteratively perform random Gibbs sampling in MCMC to converge the Markov chain.The results of simulation and experiment indicate that our estimation method achieves lower mean square error and bit error rate compared to the classic SBL approach.Additionally,it demonstrates an acceptable convergence speed.

Synthetic aperture radar imaging based on attributed scatter model using sparse recovery techniques

The sparse recovery algorithms formulate synthetic aperture radar （SAR） imaging problem in terms of sparse representation （SR） of a small number of strong scatters＇ positions among a much large number of potential scatters＇ positions, and provide an effective approach to improve the SAR image resolution. Based on the attributed scatter center model, several experiments were performed with different practical considerations to evaluate the performance of five representative SR techniques, namely, sparse Bayesian learning （SBL）, fast Bayesian matching pursuit （FBMP）, smoothed 10 norm method （SL0）, sparse reconstruction by separable approximation （SpaRSA）, fast iterative shrinkage-thresholding algorithm （FISTA）, and the parameter settings in five SR algorithms were discussed. In different situations, the performances of these algorithms were also discussed. Through the comparison of MSE and failure rate in each algorithm simulation, FBMP and SpaRSA are found suitable for dealing with problems in the SAR imaging based on attributed scattering center model. Although the SBL is time-consuming, it always get better performance when related to failure rate and high SNR.

Denoising enabled channel estimation for underwater acoustic communications:A sparsity-aware model-driven learning approach

It has always been difficult to achieve accurate information of the channel for underwater acoustic communications because of the severe underwater propagation conditions,including frequency-selective property,high relative mobility,long propagation latency,and intensive ambient noise,etc.To this end,a deep unfolding neural network based approach is proposed,in which multiple layers of the network mimic the iterations of the classical iterative sparse approximation algorithm to extract the inherent sparse features of the channel by exploiting deep learning,and a scheme based on the Sparsity-Aware DNN(SA-DNN)for UAC estimation is proposed to improve the estimation accuracy.Moreover,we propose a Denoising Sparsity-Aware DNN(DeSA-DNN)based enhanced method that integrates a denoising CNN module in the sparsity-aware deep network,so that the degradation brought by intensive ambient noise could be eliminated and the estimation accuracy can be further improved.Simulation results demonstrate that the performance of the proposed schemes is superior to the state-of-the-art compressed sensing based and iterative sparse recovery schems in the aspects of channel recovery precision,pilot overhead,and robustness,particularly under unideal circumstances of intensive ambient noise or inadequate measurement pilots.

Online identification of time-varying dynamical systems for industrial robots based on sparse Bayesian learning

Nowadays, industrial robots have been widely used in manufacturing, healthcare, packaging, and more. Choosing robots in these applications mainly attributes to their repeatability and precision. However, prolonged and loaded operations can deteriorate the accuracy and efficiency of industrial robots due to the unavoidable accumulated kinematical and dynamical errors. This paper resolves these aforementioned issues by proposing an online time-varying sparse Bayesian learning(SBL) method to identify dynamical systems of robots in real-time. The identification of dynamical systems for industrial robots is cast as a sparse linear regression problem. By constructing the dictionary matrix, the parameters of the robot dynamics are effectively estimated via a re-weighted1-minimization algorithm. Online recursive methods are integrated into SBL to achieve real-time system identification. By including sparsity and promoting online learning, the proposed method can handle time-varying dynamical systems and therefore improve operational stability and accuracy. Experimental results on both simulated and real selective compliance assembly robot arm(SCARA) robots have demonstrated the effectiveness of the proposed method for industrial robots.

On fast estimation of direction of arrival for underwater acoustic target based on sparse Bayesian learning

The Direction of Arrival （DOA） estimation methods for underwater acoustic target using Temporally Multiple Sparse Bayesian Learning （TMSBL） as the reconstructing algorithm have the disadvantage of slow computing speed. To solve this problem, a fast underwater acoustic target direction of arrival estimation was proposed. Analyzing the model characteristics of block-sparse Bayesian learning framework for DOA estimation, an algorithm was proposed to obtain the value of core hyper-parameter through MacKay＇s fixed-point method to estimate the DOA. By this process, it will spend less time for computation and provide more superior recovery performance than TMSBL algorithm. Simulation results verified the feasibility and effectiveness of the proposed algorithm.

Efficient scheme of low-dose CT reconstruction using TV minimization with an adaptive stopping strategy and sparse dictionary learning for post-processing

Recently, low-dose computed tomography （CT） has become highly desirable because of the growing concern for the potential risks of excessive radiation. For low-dose CT imaging, it is a significant challenge to guarantee image quality while reducing radiation dosage. Compared with classical filtered backprojection algorithms, compressed sensing-based iterative re- construction has achieved excellent imaging performance, but its clinical application is hindered due to its computational ineffi- ciency. To promote low-dose CT imaging, we propose a promising reconstruction scheme which combines total-variation mini- mization and sparse dictionary learning to enhance the reconstruction performance, and properly schedule them with an adaptive iteration stopping strategy to boost the reconstruction speed. Experiments conducted on a digital phantom and a physical phantom demonstrate a superior performance of our method over other methods in terms of image quality and computational efficiency, which validates its potential for low-dose CT imaging.

Data-Driven Discovery of Stochastic Differential Equations

Stochastic differential equations(SDEs)are mathematical models that are widely used to describe complex processes or phenomena perturbed by random noise from different sources.The identification of SDEs governing a system is often a challenge because of the inherent strong stochasticity of data and the complexity of the system’s dynamics.The practical utility of existing parametric approaches for identifying SDEs is usually limited by insufficient data resources.This study presents a novel framework for identifying SDEs by leveraging the sparse Bayesian learning(SBL)technique to search for a parsimonious,yet physically necessary representation from the space of candidate basis functions.More importantly,we use the analytical tractability of SBL to develop an efficient way to formulate the linear regression problem for the discovery of SDEs that requires considerably less time-series data.The effectiveness of the proposed framework is demonstrated using real data on stock and oil prices,bearing variation,and wind speed,as well as simulated data on well-known stochastic dynamical systems,including the generalized Wiener process and Langevin equation.This framework aims to assist specialists in extracting stochastic mathematical models from random phenomena in the natural sciences,economics,and engineering fields for analysis,prediction,and decision making.

DOA Estimation Based on Root Sparse Bayesian Learning Under Gain and Phase Error

The direction of arrival(DOA)is approximated by first-order Taylor expansion in most of the existing methods,which will lead to limited estimation accuracy when using coarse mesh owing to the off-grid error.In this paper,a new root sparse Bayesian learning based DOA estimation method robust to gain-phase error is proposed,which dynamically adjusts the grid angle under coarse grid spacing to compensate the off-grid error and applies the expectation maximization(EM)method to solve the respective iterative formula-based on the prior distribution of each parameter.Simulation results verify that the proposed method reduces the computational complexity through coarse grid sampling while maintaining a reasonable accuracy under gain and phase errors,as compared to the existing methods.

Data-driven discovery of linear dynamical systems from noisy data

In modern science and engineering disciplines,data-driven discovery methods play a fundamental role in system modeling,as data serve as the external representations of the intrinsic mechanisms within systems.However,empirical data contaminated by process and measurement noise remain a significant obstacle for this type of modeling.In this study,we have developed a data-driven method capable of directly uncovering linear dynamical systems from noisy data.This method combines the Kalman smoothing and sparse Bayesian learning to decouple process and measurement noise under the expectation-maximization framework,presenting an analytical method for alternate state estimation and system identification.Furthermore,the discovered model explicitly characterizes the probability distribution of process and measurement noise,as they are essential for filtering,smoothing,and stochastic control.We have successfully applied the proposed algorithm to several simulation systems.Experimental results demonstrate its potential to enable linear dynamical system discovery in practical applications where noise-free data are intractable to capture.

Combined UAMP and MF Message Passing Algorithm for Multi-Target Wideband DOA Estimation with Dirichlet Process Prior

When estimating the direction of arrival (DOA) of wideband signals from multiple sources, the performance of sparse Bayesian methods is influenced by the frequency bands occupied by signals in different directions. This is particularly true when multiple signal frequency bands overlap. Message passing algorithms (MPA) with Dirichlet process (DP) prior can be employed in a sparse Bayesian learning (SBL) framework with high precision. However, existing methods suffer from either high complexity or low precision. To address this, we propose a low-complexity DOA estimation algorithm based on a factor graph. This approach introduces two strong constraints via a stretching transformation of the factor graph. The first constraint separates the observation from the DP prior, enabling the application of the unitary approximate message passing (UAMP) algorithm for simplified inference and mitigation of divergence issues. The second constraint compensates for the deviation in estimation angle caused by the grid mismatch problem. Compared to state-of-the-art algorithms, our proposed method offers higher estimation accuracy and lower complexity.

Codimensional matrix pairing perspective of BYY harmony learning:hierarchy of bilinear systems,joint decomposition of data-covariance,and applications of network biology

One paper in a preceding issue of this journal has introduced the Bayesian Ying-Yang(BYY)harmony learning from a perspective of problem solving,parameter learning,and model selection.In a complementary role,the paper provides further insights from another perspective that a co-dimensional matrix pair(shortly co-dim matrix pair)forms a building unit and a hierarchy of such building units sets up the BYY system.The BYY harmony learning is re-examined via exploring the nature of a co-dim matrix pair,which leads to improved learning performance with refined model selection criteria and a modified mechanism that coordinates automatic model selection and sparse learning.Besides updating typical algorithms of factor analysis(FA),binary FA(BFA),binary matrix factorization(BMF),and nonnegative matrix factorization(NMF)to share such a mechanism,we are also led to(a)a new parametrization that embeds a de-noise nature to Gaussian mixture and local FA(LFA);(b)an alternative formulation of graph Laplacian based linear manifold learning;(c)a codecomposition of data and covariance for learning regularization and data integration;and(d)a co-dim matrix pair based generalization of temporal FA and state space model.Moreover,with help of a co-dim matrix pair in Hadamard product,we are led to a semi-supervised formation for regression analysis and a semi-blind learning formation for temporal FA and state space model.Furthermore,we address that these advances provide with new tools for network biology studies,including learning transcriptional regulatory,Protein-Protein Interaction network alignment,and network integration.