Transfer learning with deep sparse auto-encoder for speech emotion recognition

In order to improve the efficiency of speech emotion recognition across corpora,a speech emotion transfer learning method based on the deep sparse auto-encoder is proposed.The algorithm first reconstructs a small amount of data in the target domain by training the deep sparse auto-encoder,so that the encoder can learn the low-dimensional structural representation of the target domain data.Then,the source domain data and the target domain data are coded by the trained deep sparse auto-encoder to obtain the reconstruction data of the low-dimensional structural representation close to the target domain.Finally,a part of the reconstructed tagged target domain data is mixed with the reconstructed source domain data to jointly train the classifier.This part of the target domain data is used to guide the source domain data.Experiments on the CASIA,SoutheastLab corpus show that the model recognition rate after a small amount of data transferred reached 89.2%and 72.4%on the DNN.Compared to the training results of the complete original corpus,it only decreased by 2%in the CASIA corpus,and only 3.4%in the SoutheastLab corpus.Experiments show that the algorithm can achieve the effect of labeling all data in the extreme case that the data set has only a small amount of data tagged.

Method for denoising and reconstructing radar HRRP using modified sparse auto-encoder

A high resolution range profile(HRRP) is a summation vector of the sub-echoes of the target scattering points acquired by a wide-band radar.Generally, HRRPs obtained in a noncooperative complex electromagnetic environment are contaminated by strong noise.Effective pre-processing of the HRRP data can greatly improve the accuracy of target recognition.In this paper, a denoising and reconstruction method for HRRP is proposed based on a Modified Sparse Auto-Encoder, which is a representative non-linear model.To better reconstruct the HRRP, a sparse constraint is added to the proposed model and the sparse coefficient is calculated based on the intrinsic dimension of HRRP.The denoising of the HRRP is performed by adding random noise to the input HRRP data during the training process and fine-tuning the weight matrix through singular-value decomposition.The results of simulations showed that the proposed method can both reconstruct the signal with fidelity and suppress noise effectively, significantly outperforming other methods, especially in low Signal-to-Noise Ratio conditions.

特征增强的Sparse Transformer目标跟踪算法

针对Transformer的自注意力机制计算量大、容易被背景分心,导致有效信息抓取不足,从而降低跟踪性能的问题,提出特征增强的Sparse Transformer目标跟踪算法。基于孪生网络骨干进行特征提取;特征增强模块利用多尺度特征图生成的上下文信息,增强目标局部特征;利用Sparse Transformer的最相关特性生成目标聚焦特征,并嵌入位置编码提升跟踪定位的精度。提出的跟踪模型以端到端的方式进行训练,在OTB100,VOT2018和LaSOT等5个数据集上进行了大量实验,实验结果表明所提算法取得了较好的跟踪性能,实时跟踪速度为34帧/s。

A Two-Layer Encoding Learning Swarm Optimizer Based on Frequent Itemsets for Sparse Large-Scale Multi-Objective Optimization

Traditional large-scale multi-objective optimization algorithms(LSMOEAs)encounter difficulties when dealing with sparse large-scale multi-objective optimization problems(SLM-OPs)where most decision variables are zero.As a result,many algorithms use a two-layer encoding approach to optimize binary variable Mask and real variable Dec separately.Nevertheless,existing optimizers often focus on locating non-zero variable posi-tions to optimize the binary variables Mask.However,approxi-mating the sparse distribution of real Pareto optimal solutions does not necessarily mean that the objective function is optimized.In data mining,it is common to mine frequent itemsets appear-ing together in a dataset to reveal the correlation between data.Inspired by this,we propose a novel two-layer encoding learning swarm optimizer based on frequent itemsets(TELSO)to address these SLMOPs.TELSO mined the frequent terms of multiple particles with better target values to find mask combinations that can obtain better objective values for fast convergence.Experi-mental results on five real-world problems and eight benchmark sets demonstrate that TELSO outperforms existing state-of-the-art sparse large-scale multi-objective evolutionary algorithms(SLMOEAs)in terms of performance and convergence speed.

A Novel Clutter Suppression Algorithm for Low-Slow-Small Targets Detecting Based on Sparse Adaptive Filtering

Passive detection of low-slow-small(LSS)targets is easily interfered by direct signal and multipath clutter,and the traditional clutter suppression method has the contradiction between step size and convergence rate.In this paper,a frequency domain clutter suppression algorithm based on sparse adaptive filtering is proposed.The pulse compression operation between the error signal and the input reference signal is added to the cost function as a sparsity constraint,and the criterion for filter weight updating is improved to obtain a purer echo signal.At the same time,the step size and penalty factor are brought into the adaptive iteration process,and the input data is used to drive the adaptive changes of parameters such as step size.The proposed algorithm has a small amount of calculation,which improves the robustness to parameters such as step size,reduces the weight error of the filter and has a good clutter suppression performance.

Enhancing Evolutionary Algorithms With Pattern Mining for Sparse Large-Scale Multi-Objective Optimization Problems

Sparse large-scale multi-objective optimization problems(SLMOPs)are common in science and engineering.However,the large-scale problem represents the high dimensionality of the decision space,requiring algorithms to traverse vast expanse with limited computational resources.Furthermore,in the context of sparse,most variables in Pareto optimal solutions are zero,making it difficult for algorithms to identify non-zero variables efficiently.This paper is dedicated to addressing the challenges posed by SLMOPs.To start,we introduce innovative objective functions customized to mine maximum and minimum candidate sets.This substantial enhancement dramatically improves the efficacy of frequent pattern mining.In this way,selecting candidate sets is no longer based on the quantity of nonzero variables they contain but on a higher proportion of nonzero variables within specific dimensions.Additionally,we unveil a novel approach to association rule mining,which delves into the intricate relationships between non-zero variables.This novel methodology aids in identifying sparse distributions that can potentially expedite reductions in the objective function value.We extensively tested our algorithm across eight benchmark problems and four real-world SLMOPs.The results demonstrate that our approach achieves competitive solutions across various challenges.

Contribution Tracking Feature Selection (CTFS) Based on the Fusion of Sparse Autoencoder and Mutual Information

For data mining tasks on large-scale data,feature selection is a pivotal stage that plays an important role in removing redundant or irrelevant features while improving classifier performance.Traditional wrapper feature selection methodologies typically require extensive model training and evaluation,which cannot deliver desired outcomes within a reasonable computing time.In this paper,an innovative wrapper approach termed Contribution Tracking Feature Selection(CTFS)is proposed for feature selection of large-scale data,which can locate informative features without population-level evolution.In other words,fewer evaluations are needed for CTFS compared to other evolutionary methods.We initially introduce a refined sparse autoencoder to assess the prominence of each feature in the subsequent wrapper method.Subsequently,we utilize an enhanced wrapper feature selection technique that merges Mutual Information(MI)with individual feature contributions.Finally,a fine-tuning contribution tracking mechanism discerns informative features within the optimal feature subset,operating via a dominance accumulation mechanism.Experimental results for multiple classification performance metrics demonstrate that the proposed method effectively yields smaller feature subsets without degrading classification performance in an acceptable runtime compared to state-of-the-art algorithms across most large-scale benchmark datasets.

THE STABLE RECONSTRUCTION OF STRONGLY-DECAYING BLOCK SPARSE SIGNALS

In this paper,we reconstruct strongly-decaying block sparse signals by the block generalized orthogonal matching pursuit(BgOMP)algorithm in the l2-bounded noise case.Under some restraints on the minimum magnitude of the nonzero elements of the strongly-decaying block sparse signal,if the sensing matrix satisfies the the block restricted isometry property(block-RIP),then arbitrary strongly-decaying block sparse signals can be accurately and steadily reconstructed by the BgOMP algorithm in iterations.Furthermore,we conjecture that this condition is sharp.

Balance Sparse Decomposition Method with Nonconvex Regularization for Gearbox Fault Diagnosis

As an important part of rotating machinery,gearboxes often fail due to their complex working conditions and harsh working environment.Therefore,it is very necessary to effectively extract the fault features of the gearboxes.Gearbox fault signals usually contain multiple characteristic components and are accompanied by strong noise interference.Traditional sparse modeling methods are based on synthesis models,and there are few studies on analysis and balance models.In this paper,a balance nonconvex regularized sparse decomposition method is proposed,which based on a balance model and an arctangent nonconvex penalty function.The sparse dictionary is constructed by using Tunable Q-Factor Wavelet Transform(TQWT)that satisfies the tight frame condition,which can achieve efficient and fast solution.It is optimized and solved by alternating direction method of multipliers(ADMM)algorithm,and the non-convex regularized sparse decomposition algorithm of synthetic and analytical models are given.Through simulation experiments,the determination methods of regularization parameters and balance parameters are given,and compared with the L1 norm regularization sparse decomposition method under the three models.Simulation analysis and engineering experimental signal analysis verify the effectiveness and superiority of the proposed method.

Sparse Reconstructive Evidential Clustering for Multi-View Data

Although many multi-view clustering(MVC) algorithms with acceptable performances have been presented, to the best of our knowledge, nearly all of them need to be fed with the correct number of clusters. In addition, these existing algorithms create only the hard and fuzzy partitions for multi-view objects,which are often located in highly-overlapping areas of multi-view feature space. The adoption of hard and fuzzy partition ignores the ambiguity and uncertainty in the assignment of objects, likely leading to performance degradation. To address these issues, we propose a novel sparse reconstructive multi-view evidential clustering algorithm(SRMVEC). Based on a sparse reconstructive procedure, SRMVEC learns a shared affinity matrix across views, and maps multi-view objects to a 2-dimensional humanreadable chart by calculating 2 newly defined mathematical metrics for each object. From this chart, users can detect the number of clusters and select several objects existing in the dataset as cluster centers. Then, SRMVEC derives a credal partition under the framework of evidence theory, improving the fault tolerance of clustering. Ablation studies show the benefits of adopting the sparse reconstructive procedure and evidence theory. Besides,SRMVEC delivers effectiveness on benchmark datasets by outperforming some state-of-the-art methods.

Cambricon-QR:a sparse and bitwise reproducible quantized training accelerator

Quantized training has been proven to be a prominent method to achieve deep neural network training under limited computational resources.It uses low bit-width arithmetics with a proper scaling factor to achieve negligible accuracy loss.Cambricon-Q is the ASIC design proposed to efficiently support quantized training,and achieves significant performance improvement.However,there are still two caveats in the design.First,Cambricon-Q with different hardware specifications may lead to different numerical errors,resulting in non-reproducible behaviors which may become a major concern in critical applications.Second,Cambricon-Q cannot leverage data sparsity,where considerable cycles could still be squeezed out.To address the caveats,the acceleration core of Cambricon-Q is redesigned to support fine-grained irregular data processing.The new design not only enables acceleration on sparse data,but also enables performing local dynamic quantization by contiguous value ranges(which is hardware independent),instead of contiguous addresses(which is dependent on hardware factors).Experimental results show that the accuracy loss of the method still keeps negligible,and the accelerator achieves 1.61×performance improvement over Cambricon-Q,with about 10%energy increase.

Improved Variable Forgetting Factor Proportionate RLS Algorithm with Sparse Penalty and Fast Implementation Using DCD Iterations

The proportionate recursive least squares(PRLS)algorithm has shown faster convergence and better performance than both proportionate updating(PU)mechanism based least mean squares(LMS)algorithms and RLS algorithms with a sparse regularization term.In this paper,we propose a variable forgetting factor(VFF)PRLS algorithm with a sparse penalty,e.g.,l_(1)-norm,for sparse identification.To reduce the computation complexity of the proposed algorithm,a fast implementation method based on dichotomous coordinate descent(DCD)algorithm is also derived.Simulation results indicate superior performance of the proposed algorithm.

SRMD:Sparse Random Mode Decomposition

Signal decomposition and multiscale signal analysis provide many useful tools for timefrequency analysis.We proposed a random feature method for analyzing time-series data by constructing a sparse approximation to the spectrogram.The randomization is both in the time window locations and the frequency sampling,which lowers the overall sampling and computational cost.The sparsification of the spectrogram leads to a sharp separation between time-frequency clusters which makes it easier to identify intrinsic modes,and thus leads to a new data-driven mode decomposition.The applications include signal representation,outlier removal,and mode decomposition.On benchmark tests,we show that our approach outperforms other state-of-the-art decomposition methods.

Sparse antenna array design methodologies:A review

Designing a sparse array with reduced transmit/receive modules(TRMs)is vital for some applications where the antenna system’s size,weight,allowed operating space,and cost are limited.Sparse arrays exhibit distinct architectures,roughly classified into three categories:Thinned arrays,nonuniformly spaced arrays,and clustered arrays.While numerous advanced synthesis methods have been presented for the three types of sparse arrays in recent years,a comprehensive review of the latest development in sparse array synthesis is lacking.This work aims to fill this gap by thoroughly summarizing these techniques.The study includes synthesis examples to facilitate a comparative analysis of different techniques in terms of both accuracy and efficiency.Thus,this review is intended to assist researchers and engineers in related fields,offering a clear understanding of the development and distinctions among sparse array synthesis techniques.

A strategy for out-of-roundness damage wheels identification in railway vehicles based on sparse autoencoders

Wayside monitoring is a promising cost-effective alternative to predict damage in the rolling stock. The main goal of this work is to present an unsupervised methodology to identify out-of-roundness(OOR) damage wheels, such as wheel flats and polygonal wheels. This automatic damage identification algorithm is based on the vertical acceleration evaluated on the rails using a virtual wayside monitoring system and involves the application of a two-step procedure. The first step aims to define a confidence boundary by using(healthy) measurements evaluated on the rail constituting a baseline. The second step of the procedure involves classifying damage of predefined scenarios with different levels of severities. The proposed procedure is based on a machine learning methodology and includes the following stages:(1) data collection,(2) damage-sensitive feature extraction from the acquired responses using a neural network model, i.e., the sparse autoencoder(SAE),(3) data fusion based on the Mahalanobis distance, and(4) unsupervised feature classification by implementing outlier and cluster analysis. This procedure considers baseline responses at different speeds and rail irregularities to train the SAE model. Then, the trained SAE is capable to reconstruct test responses(not trained) allowing to compute the accumulative difference between original and reconstructed signals. The results prove the efficiency of the proposed approach in identifying the two most common types of OOR in railway wheels.

Impact Force Localization and Reconstruction via ADMM-based Sparse Regularization Method

In practice,simultaneous impact localization and time history reconstruction can hardly be achieved,due to the illposed and under-determined problems induced by the constrained and harsh measuring conditions.Although l_(1) regularization can be used to obtain sparse solutions,it tends to underestimate solution amplitudes as a biased estimator.To address this issue,a novel impact force identification method with l_(p) regularization is proposed in this paper,using the alternating direction method of multipliers(ADMM).By decomposing the complex primal problem into sub-problems solvable in parallel via proximal operators,ADMM can address the challenge effectively.To mitigate the sensitivity to regularization parameters,an adaptive regularization parameter is derived based on the K-sparsity strategy.Then,an ADMM-based sparse regularization method is developed,which is capable of handling l_(p) regularization with arbitrary p values using adaptively-updated parameters.The effectiveness and performance of the proposed method are validated on an aircraft skin-like composite structure.Additionally,an investigation into the optimal p value for achieving high-accuracy solutions via l_(p) regularization is conducted.It turns out that l_(0.6)regularization consistently yields sparser and more accurate solutions for impact force identification compared to the classic l_(1) regularization method.The impact force identification method proposed in this paper can simultaneously reconstruct impact time history with high accuracy and accurately localize the impact using an under-determined sensor configuration.

Three-dimensional pseudo-dynamic reliability analysis of seismic shield tunnel faces combined with sparse polynomial chaos expansion

To address the seismic face stability challenges encountered in urban and subsea tunnel construction,an efficient probabilistic analysis framework for shield tunnel faces under seismic conditions is proposed.Based on the upper-bound theory of limit analysis,an improved three-dimensional discrete deterministic mechanism,accounting for the heterogeneous nature of soil media,is formulated to evaluate seismic face stability.The metamodel of failure probabilistic assessments for seismic tunnel faces is constructed by integrating the sparse polynomial chaos expansion method(SPCE)with the modified pseudo-dynamic approach(MPD).The improved deterministic model is validated by comparing with published literature and numerical simulations results,and the SPCE-MPD metamodel is examined with the traditional MCS method.Based on the SPCE-MPD metamodels,the seismic effects on face failure probability and reliability index are presented and the global sensitivity analysis(GSA)is involved to reflect the influence order of seismic action parameters.Finally,the proposed approach is tested to be effective by a engineering case of the Chengdu outer ring tunnel.The results show that higher uncertainty of seismic response on face stability should be noticed in areas with intense earthquakes and variation of seismic wave velocity has the most profound influence on tunnel face stability.

THE SPARSE REPRESENTATION RELATED WITH FRACTIONAL HEAT EQUATIONS

This study introduces a pre-orthogonal adaptive Fourier decomposition(POAFD)to obtain approximations and numerical solutions to the fractional Laplacian initial value problem and the extension problem of Caffarelli and Silvestre(generalized Poisson equation).As a first step,the method expands the initial data function into a sparse series of the fundamental solutions with fast convergence,and,as a second step,makes use of the semigroup or the reproducing kernel property of each of the expanding entries.Experiments show the effectiveness and efficiency of the proposed series solutions.

Adaptive Sparse Grid Discontinuous Galerkin Method:Review and Software Implementation

This paper reviews the adaptive sparse grid discontinuous Galerkin(aSG-DG)method for computing high dimensional partial differential equations(PDEs)and its software implementation.The C++software package called AdaM-DG,implementing the aSG-DG method,is available on GitHub at https://github.com/JuntaoHuang/adaptive-multiresolution-DG.The package is capable of treating a large class of high dimensional linear and nonlinear PDEs.We review the essential components of the algorithm and the functionality of the software,including the multiwavelets used,assembling of bilinear operators,fast matrix-vector product for data with hierarchical structures.We further demonstrate the performance of the package by reporting the numerical error and the CPU cost for several benchmark tests,including linear transport equations,wave equations,and Hamilton-Jacobi(HJ)equations.

Design and implementation of code acquisition using sparse Fourier transform

Code acquisition is the kernel operation for signal synchronization in the spread-spectrum receiver.To reduce the computational complexity and latency of code acquisition,this paper proposes an efficient scheme employing sparse Fourier transform(SFT)and the relevant hardware architecture for field programmable gate array(FPGA)and application-specific integrated circuit(ASIC)implementation.Efforts are made at both the algorithmic level and the implementation level to enable merged searching of code phase and Doppler frequency without incurring massive hardware expenditure.Compared with the existing code acquisition approaches,it is shown from theoretical analysis and experimental results that the proposed design can shorten processing latency and reduce hardware complexity without degrading the acquisition probability.