Feature Extraction of Kernel Regress Reconstruction for Fault Diagnosis Based on Self-organizing Manifold Learning

The feature space extracted from vibration signals with various faults is often nonlinear and of high dimension.Currently,nonlinear dimensionality reduction methods are available for extracting low-dimensional embeddings,such as manifold learning.However,these methods are all based on manual intervention,which have some shortages in stability,and suppressing the disturbance noise.To extract features automatically,a manifold learning method with self-organization mapping is introduced for the first time.Under the non-uniform sample distribution reconstructed by the phase space,the expectation maximization(EM) iteration algorithm is used to divide the local neighborhoods adaptively without manual intervention.After that,the local tangent space alignment(LTSA) algorithm is adopted to compress the high-dimensional phase space into a more truthful low-dimensional representation.Finally,the signal is reconstructed by the kernel regression.Several typical states include the Lorenz system,engine fault with piston pin defect,and bearing fault with outer-race defect are analyzed.Compared with the LTSA and continuous wavelet transform,the results show that the background noise can be fully restrained and the entire periodic repetition of impact components is well separated and identified.A new way to automatically and precisely extract the impulsive components from mechanical signals is proposed.

Software Defect Prediction Based on Non-Linear Manifold Learning and Hybrid Deep Learning Techniques

Software defect prediction plays a very important role in software quality assurance,which aims to inspect as many potentially defect-prone software modules as possible.However,the performance of the prediction model is susceptible to high dimensionality of the dataset that contains irrelevant and redundant features.In addition,software metrics for software defect prediction are almost entirely traditional features compared to the deep semantic feature representation from deep learning techniques.To address these two issues,we propose the following two solutions in this paper:(1)We leverage a novel non-linear manifold learning method-SOINN Landmark Isomap(SL-Isomap)to extract the representative features by selecting automatically the reasonable number and position of landmarks,which can reveal the complex intrinsic structure hidden behind the defect data.(2)We propose a novel defect prediction model named DLDD based on hybrid deep learning techniques,which leverages denoising autoencoder to learn true input features that are not contaminated by noise,and utilizes deep neural network to learn the abstract deep semantic features.We combine the squared error loss function of denoising autoencoder with the cross entropy loss function of deep neural network to achieve the best prediction performance by adjusting a hyperparameter.We compare the SL-Isomap with seven state-of-the-art feature extraction methods and compare the DLDD model with six baseline models across 20 open source software projects.The experimental results verify that the superiority of SL-Isomap and DLDD on four evaluation indicators.

Adaptive Neighboring Selection Algorithm Based on Curvature Prediction in Manifold Learning

Recently manifold learning algorithm for dimensionality reduction attracts more and more interests, and various linear and nonlinear,global and local algorithms are proposed. The key step of manifold learning algorithm is the neighboring region selection. However,so far for the references we know,few of which propose a generally accepted algorithm to well select the neighboring region. So in this paper,we propose an adaptive neighboring selection algorithm,which successfully applies the LLE and ISOMAP algorithms in the test. It is an algorithm that can find the optimal K nearest neighbors of the data points on the manifold. And the theoretical basis of the algorithm is the approximated curvature of the data point on the manifold. Based on Riemann Geometry,Jacob matrix is a proper mathematical concept to predict the approximated curvature. By verifying the proposed algorithm on embedding Swiss roll from R3 to R2 based on LLE and ISOMAP algorithm,the simulation results show that the proposed adaptive neighboring selection algorithm is feasible and able to find the optimal value of K,making the residual variance relatively small and better visualization of the results. By quantitative analysis,the embedding quality measured by residual variance is increased 45. 45% after using the proposed algorithm in LLE.

Indoor localization via l^1-graph regularized semi-supervised manifold learning

In this paper, a new -graph regularized semi-supervised manifold learning （LRSML） method is proposed for indoor localization. Due to noise corruption and non-linearity of received signal strength （RSS）, traditional approaches always fail to deliver accurate positioning results. The -graph is constructed by sparse representation of each sample with respect to remaining samples. Noise factor is considered in the construction process of -graph, leading to more robustness compared to traditional k-nearest-neighbor graph （KNN-graph）. The KNN-graph construction is supervised, while the -graph is assumed to be unsupervised without harnessing any data label information and uncovers the underlying sparse relationship of each data. Combining KNN-graph and -graph, both labeled and unlabeled information are utilized, so the LRSML method has the potential to convey more discriminative information compared to conventional methods. To overcome the non-linearity of RSS, kernel-based manifold learning method （K-LRSML） is employed through mapping the original signal data to a higher dimension Hilbert space. The efficiency and superiority of LRSML over current state of art methods are verified with extensive experiments on real data.

Autonomic failure prediction based on manifold learning for large-scale distributed systems

This article investigates autonomic failure prediction in large-scale distributed systems with nonlinear dimensionality reduction to automatically extract failure features. Most existing methods for failure prediction focus on building prediction models or heuristic rules by discovering failure patterns, but the process of feature extraction before failure patterns recognition is rarely considered due to the increasing complexity of modern distributed systems. In this work, a novel performance-centric approach to automate failure prediction is proposed based on manifold learning （ML）. In addition, the ML algorithm named supervised locally linear embedding （SLLE） is applied to achieve feature extraction. To generalize the dimensionality reduction mapping, the nonlinear mapping approximation and optimization solution is also proposed. In experimental work a file transfer test bed with fault injection is developed which can gather multilevel performance metrics transparently. Based on the runtime monitoring of these metrics, the SLLE method can automatically predict more than 50% of the central processing unit （CPU） and memory failures, and around 70% of the network failure.

Research of Adaptive Neighborhood Incremental Principal Component Analysis and Locality Preserving Projection Manifold Learning Algorithm

In view of the incremental learning problem of manifold learning algorithm, an adaptive neighborhood incremental principal component analysis(PCA) and locality preserving projection(LPP) manifold learning algorithm is presented, and the incremental learning principle of algorithm is introduced. For incremental sample data, the adjacency and covariance matrices are incrementally updated by the existing samples; then the dimensionality reduction results of the incremental samples are estimated by the dimensionality reduction results of the existing samples; finally, the dimensionality reduction results of the incremental and existing samples are updated by subspace iteration method. The adaptive neighborhood incremental PCA-LPP manifold learning algorithm is applied to processing of gearbox fault signals. The dimensionality reduction results by incremental learning have very small error, compared with those by batch learning. Spatial aggregation of the incremental samples is basically stable, and fault identification rate is increased.

Unsupervised Nonlinear Adaptive Manifold Learning for Global and Local Information

In this paper,we propose an Unsupervised Nonlinear Adaptive Manifold Learning method(UNAML)that considers both global and local information.In this approach,we apply unlabeled training samples to study nonlinear manifold features,while considering global pairwise distances and maintaining local topology structure.Our method aims at minimizing global pairwise data distance errors as well as local structural errors.In order to enable our UNAML to be more efficient and to extract manifold features from the external source of new data,we add a feature approximate error that can be used to learn a linear extractor.Also,we add a feature approximate error that can be used to learn a linear extractor.In addition,we use a method of adaptive neighbor selection to calculate local structural errors.This paper uses the kernel matrix method to optimize the original algorithm.Our algorithm proves to be more effective when compared with the experimental results of other feature extraction methods on real face-data sets and object data sets.

Incremental Alignment Manifold Learning

A new manifold learning method, called incremental alignment method （IAM）, is proposed for nonlinear dimensionality reduction of high dimensional data with intrinsic low dimensionality. The main idea is to incrementally align low-dimensional coordinates of input data patch-by-patch to iteratively generate the representation of the entire data.set. The method consists of two major steps, the incremental step and the alignment step. The incremental step incrementally searches neighborhood patch to be aligned in the next step, and the alignment step iteratively aligns the low-dimensional coordinates of the neighborhood patch searched to generate the embeddings of the entire dataset. Compared with the existing manifold learning methods, the proposed method dominates in several aspects： high efficiency, easy out-of-sample extension, well metric-preserving, and averting of the local minima issue. All these properties are supported by a series of experiments performed on the synthetic and real-life datasets. In addition, the computational complexity of the proposed method is analyzed, and its efficiency is theoretically argued and experimentally demonstrated.

Facial Expression Recognition of Various Internal States via Manifold Learning

Emotions are becoming increasingly important in human-centered interaction architectures. Recognition of facial expressions, which are central to human-computer interactions, seems natural and desirable. However, facial expressions include mixed emotions, continuous rather than discrete, which vary from moment to moment. This paper represents a novel method of recognizing facial expressions of various internal states via manifold learning, to achieve the aim of humancentered interaction studies. A critical review of widely used emotion models is described, then, facial expression features of various internal states via the locally linear embedding （LLE） are extracted. The recognition of facial expressions is created with the pleasure-displeasure and arousal-sleep dimensions in a two-dimensional model of emotion. The recognition result of various internal state expressions that mapped to the embedding space via the LLE algorithm can effectively represent the structural nature of the two-dimensional model of emotion. Therefore our research has established that the relationship between facial expressions of various internal states can be elaborated in the two-dimensional model of emotion, via the locally linear embedding algorithm.

Atlas Compatibility Transformation:A Normal Manifold Learning Algorithm

Over the past few years,nonlinear manifold learning has been widely exploited in data analysis and machine learning.This paper presents a novel manifold learning algorithm,named atlas compatibility transformation(ACT),It solves two problems which correspond to two key points in the manifold definition:how to chart a given manifold and how to align the patches to a global coordinate space based on compatibility.For the first problem,we divide the manifold into maximal linear patch(MLP) based on normal vector field of the manifold.For the second problem,we align patches into an optimal global system by solving a generalized eigenvalue problem.Compared with the traditional method,the ACT could deal with noise datasets and fragment datasets.Moreover,the mappings between high dimensional space and low dimensional space are given.Experiments on both synthetic data and real-world data indicate the effection of the proposed algorithm.

Alternating minimization for data-driven computational elasticity from experimental data: kernel method for learning constitutive manifold

Data-driven computing in elasticity attempts to directly use experimental data on material,without constructing an empirical model of the constitutive relation,to predict an equilibrium state of a structure subjected to a specified external load.Provided that a data set comprising stress-strain pairs of material is available,a data-driven method using the kernel method and the regularized least-squares was developed to extract a manifold on which the points in the data set approximately lie(Kanno 2021,Jpn.J.Ind.Appl.Math.).From the perspective of physical experiments,stress field cannot be directly measured,while displacement and force fields are measurable.In this study,we extend the previous kernel method to the situation that pairs of displacement and force,instead of pairs of stress and strain,are available as an input data set.A new regularized least-squares problem is formulated in this problem setting,and an alternating minimization algorithm is proposed to solve the problem.

An improved Isomap method for manifold learning

Purpose-Isometric feature mapping(Isomap)is a very popular manifold learning method and is widely used in dimensionality reduction and data visualization.The most time-consuming step in Isomap is to compute the shortest paths between all pairs of data points based on a neighbourhood graph.The classical Isomap(C-Isomap)is very slow,due to the use of Floyd’s algorithm to compute the shortest paths.The purpose of this paper is to speed up Isomap.Design/methodology/approach-Through theoretical analysis,it is found that the neighbourhood graph in Isomap is sparse.In this case,the Dijkstra’s algorithm with Fibonacci heap(Fib-Dij)is faster than Floyd’s algorithm.In this paper,an improved Isomap method based on Fib-Dij is proposed.By using Fib-Dij to replace Floyd’s algorithm,an improved Isomap method is presented in this paper.Findings-Using the S-curve,the Swiss-roll,the Frey face database,the mixed national institute of standards and technology database of handwritten digits and a face image database,the performance of the proposed method is compared with C-Isomap,showing the consistency with C-Isomap and marked improvements in terms of the high speed.Simulations also demonstrate that Fib-Dij reduces the computation time of the shortest paths from O(N3)to O(N2lgN).Research limitations/implications-Due to the limitations of the computer,the sizes of the data sets in this paper are all smaller than 3,000.Therefore,researchers are encouraged to test the proposed algorithm on larger data sets.Originality/value-The new method based on Fib-Dij can greatly improve the speed of Isomap.

Semi-Supervised Learning Based on Manifold in BCI

A Laplacian support vector machine （LapSVM） algorithm, a semi-supervised learning based on manifold, is introduced to brain-computer interface （BCI） to raise the classification precision and reduce the subjects＇ training complexity. The data are collected from three subjects in a three-task mental imagery experiment. LapSVM and transductive SVM （TSVM） are trained with a few labeled samples and a large number of unlabeled samples. The results confirm that LapSVM has a much better classification than TSVM.

Manifold Structure Analysis of Tactical Network Traffic Matrix Based on Maximum Variance Unfolding Algorithm

As modern weapons and equipment undergo increasing levels of informatization,intelligence,and networking,the topology and traffic characteristics of battlefield data networks built with tactical data links are becoming progressively complex.In this paper,we employ a traffic matrix to model the tactical data link network.We propose a method that utilizes the Maximum Variance Unfolding(MVU)algorithm to conduct nonlinear dimensionality reduction analysis on high-dimensional open network traffic matrix datasets.This approach introduces novel ideas and methods for future applications,including traffic prediction and anomaly analysis in real battlefield network environments.

Improved Multi-Bandwidth Mode Manifold for Enhanced Bearing Fault Diagnosis

Variational mode decomposition(VMD) has been proved to be useful for extraction of fault-induced transients of rolling bearings. Multi-bandwidth mode manifold(Triple M, TM) is one variation of the VMD, which units multiple fault-related modes with different bandwidths by a nonlinear manifold learning algorithm named local tangent space alignment(LTSA). The merit of the TM method is that the bearing fault-induced transients extracted contain low level of in-band noise without optimization of the VMD parameters. However, the determination of the neighborhood size of the LTSA is time-consuming, and the extracted fault-induced transients may have the problem of asymmetry in the up-and-down direction. This paper aims to improve the efficiency and waveform symmetry of the TM method.Specifically, the multi-bandwidth modes consisting of the fault-related modes with different bandwidths are first obtained by repeating the recycling VMD(RVMD) method with different bandwidth balance parameters. Then, the LTSA algorithm is performed on the multi-bandwidth modes to extract their inherent manifold structure, in which the natural nearest neighbor(Triple N, TN) algorithm is adopted to efficiently and reasonably select the neighbors of each data point in the multi-bandwidth modes. Finally, a weight-based feature compensation strategy is designed to synthesize the low-dimensional manifold features to alleviate the asymmetry problem, resulting in a symmetric TM feature that can represent the real fault transient components. The major contribution of the improved TM method for bearing fault diagnosis is that the pure fault-induced transients are extracted efficiently and are symmetrical as the real. One simulation analysis and two experimental applications in bearing fault diagnosis validate the enhanced performance of the improved TM method over the traditional methods. This research proposes a bearing fault diagnosis method which has the advantages of high efficiency, good waveform symmetry and enhanced in-band noise removal capability.

MODIFIED LAPLACIAN EIGENMAP ETHOD FOR FAULT DIAGNOSIS

A novel method based on the improved Laplacian eigenmap algorithm for fault pattern classification is proposed. Via modifying the Laplacian eigenmap algorithm to replace Euclidean distance with kernel-based geometric distance in the neighbor graph construction, the method can preserve the consistency of local neighbor information and effectively extract the low-dimensional manifold features embedded in the high-dimensional nonlinear data sets. A nonlinear dimensionality reduction algorithm based on the improved Laplacian eigenmap is to directly learn high-dimensional fault signals and extract the intrinsic manifold features from them. The method greatly preserves the global geometry structure information embedded in the signals, and obviously improves the classification performance of fault pattern recognition. The experimental results on both simulation and engineering indicate the feasibility and effectiveness of the new method.

Fault Diagnosis Model Based on Feature Compression with Orthogonal Locality Preserving Projection

Based on feature compression with orthogonal locality preserving projection（OLPP）,a novel fault diagnosis model is proposed in this paper to achieve automation and high-precision of fault diagnosis of rotating machinery.With this model,the original vibration signals of training and test samples are first decomposed through the empirical mode decomposition（EMD）,and Shannon entropy is constructed to achieve high-dimensional eigenvectors.In order to replace the traditional feature extraction way which does the selection manually,OLPP is introduced to automatically compress the high-dimensional eigenvectors of training and test samples into the low-dimensional eigenvectors which have better discrimination.After that,the low-dimensional eigenvectors of training samples are input into Morlet wavelet support vector machine（MWSVM） and a trained MWSVM is obtained.Finally,the low-dimensional eigenvectors of test samples are input into the trained MWSVM to carry out fault diagnosis.To evaluate our proposed model,the experiment of fault diagnosis of deep groove ball bearings is made,and the experiment results indicate that the recognition accuracy rate of the proposed diagnosis model for outer race crack、inner race crack and ball crack is more than 90%.Compared to the existing approaches,the proposed diagnosis model combines the strengths of EMD in fault feature extraction,OLPP in feature compression and MWSVM in pattern recognition,and realizes the automation and high-precision of fault diagnosis.

Super-Resolution for Face Image with an Improved K-NN Search Strategy

Recently, neighbor embedding based face super-resolution(SR) methods have shown the ability for achieving high-quality face images, those methods are based on the assumption that the same neighborhoods are preserved in both low-resolution(LR) training set and high-resolution(HR) training set. However, due to the "one-to-many" mapping between the LR image and HR ones in practice, the neighborhood relationship of the LR patch in LR space is quite different with that of the HR counterpart, that is to say the neighborhood relationship obtained is not true. In this paper, we explore a novel and effective re-identified K-nearest neighbor(RIKNN) method to search neighbors of LR patch. Compared with other methods, our method uses the geometrical information of LR manifold and HR manifold simultaneously. In particular, it searches K-NN of LR patch in the LR space and refines the searching results by re-identifying in the HR space, thus giving rise to accurate K-NN and improved performance. A statistical analysis of the influence of the training set size and nearest neighbor number is given, experimental results on some public face databases show the superiority of our proposed scheme over state-of-the-art face hallucination approaches in terms of subjective and objective results as well as computational complexity.

LLE-BASED CLASSIFICATION ALGORITHM FOR MMW RADAR TARGET RECOGNITION

In this paper,a new multiclass classification algorithm is proposed based on the idea of Locally Linear Embedding(LLE),to avoid the defect of traditional manifold learning algorithms,which can not deal with new sample points.The algorithm defines an error as a criterion by computing a sample's reconstruction weight using LLE.Furthermore,the existence and characteristics of low dimensional manifold in range-profile time-frequency information are explored using manifold learning algorithm,aiming at the problem of target recognition about high range resolution MilliMeter-Wave(MMW) radar.The new algorithm is applied to radar target recognition.The experiment results show the algorithm is efficient.Compared with other classification algorithms,our method improves the recognition precision and the result is not sensitive to input parameters.

Locality Preserving Discriminant Projection for Speaker Verification

In this paper, a manifold subspace learning algorithm based on locality preserving discriminant projection (LPDP) is used for speaker verification. LPDP can overcome the deficiency of the total variability factor analysis and locality preserving projection (LPP). LPDP can effectively use the speaker label information of speech data. Through optimization, LPDP can maintain the inherent manifold local structure of the speech data samples of the same speaker by reducing the distance between them. At the same time, LPDP can enhance the discriminability of the embedding space by expanding the distance between the speech data samples of different speakers. The proposed method is compared with LPP and total variability factor analysis on the NIST SRE 2010 telephone-telephone core condition. The experimental results indicate that the proposed LPDP can overcome the deficiency of LPP and total variability factor analysis and can further improve the system performance.