Kernel principal component analysis network for image classification

In order to classify nonlinear features with a linear classifier and improve the classification accuracy, a deep learning network named kernel principal component analysis network（ KPCANet） is proposed. First, the data is mapped into a higher-dimensional space with kernel principal component analysis to make the data linearly separable. Then a two-layer KPCANet is built to obtain the principal components of the image. Finally, the principal components are classified with a linear classifier. Experimental results showthat the proposed KPCANet is effective in face recognition, object recognition and handwritten digit recognition. It also outperforms principal component analysis network（ PCANet） generally. Besides, KPCANet is invariant to illumination and stable to occlusion and slight deformation.

Feature Fusion Multi-View Hashing Based on Random Kernel Canonical Correlation Analysis

Hashing technology has the advantages of reducing data storage and improving the efficiency of the learning system,making it more and more widely used in image retrieval.Multi-view data describes image information more comprehensively than traditional methods using a single-view.How to use hashing to combine multi-view data for image retrieval is still a challenge.In this paper,a multi-view fusion hashing method based on RKCCA(Random Kernel Canonical Correlation Analysis)is proposed.In order to describe image content more accurately,we use deep learning dense convolutional network feature DenseNet to construct multi-view by combining GIST feature or BoW_SIFT(Bag-of-Words model+SIFT feature)feature.This algorithm uses RKCCA method to fuse multi-view features to construct association features and apply them to image retrieval.The algorithm generates binary hash code with minimal distortion error by designing quantization regularization terms.A large number of experiments on benchmark datasets show that this method is superior to other multi-view hashing methods.

Cobalt crust recognition based on kernel Fisher discriminant analysis and genetic algorithm in reverberation environment

Recognition of substrates in cobalt crust mining areas can improve mining efficiency.Aiming at the problem of unsatisfactory performance of using single feature to recognize the seabed material of the cobalt crust mining area,a method based on multiple-feature sets is proposed.Features of the target echoes are extracted by linear prediction method and wavelet analysis methods,and the linear prediction coefficient and linear prediction cepstrum coefficient are also extracted.Meanwhile,the characteristic matrices of modulus maxima,sub-band energy and multi-resolution singular spectrum entropy are obtained,respectively.The resulting features are subsequently compressed by kernel Fisher discriminant analysis(KFDA),the output features are selected using genetic algorithm(GA)to obtain optimal feature subsets,and recognition results of classifier are chosen as genetic fitness function.The advantages of this method are that it can describe the signal features more comprehensively and select the favorable features and remove the redundant features to the greatest extent.The experimental results show the better performance of the proposed method in comparison with only using KFDA or GA.

Face Recognition Using Kernel Discriminant Analysis

Linear discrimiant analysis (LDA) has been used in face recognition. But it is difficult to handle the high nonlinear problems, such as changes of large viewpoint and illumination. In order to overcome these problems, kernel discriminant analysis for face recognition is presented. This approach adopts the kernel functions to replace the dot products of nonlinear mapping in the high dimensional feature space, and then the nonlinear problem can be solved in the input space conveniently without explicit mapping. Two face databases are given.

Kernel Model Applied in Kernel Direct Discriminant Analysis for the Recognition of Face with Nonlinear Variations

A kernel-based discriminant analysis method called kernel direct discriminant analysis is employed, which combines the merit of direct linear discriminant analysis with that of kernel trick. In order to demonstrate its better robustness to the complex and nonlinear variations of real face images, such as illumination, facial expression, scale and pose variations, experiments are carried out on the Olivetti Research Laboratory, Yale and self-built face databases. The results indicate that in contrast to kernel principal component analysis and kernel linear discriminant analysis, the method can achieve lower （7%） error rate using only a very small set of features. Furthermore, a new corrected kernel model is proposed to improve the recognition performance. Experimental results confirm its superiority （1% in terms of recognition rate） to other polynomial kernel models.

A deep kernel method for lithofacies identification using conventional well logs

How to fit a properly nonlinear classification model from conventional well logs to lithofacies is a key problem for machine learning methods.Kernel methods(e.g.,KFD,SVM,MSVM)are effective attempts to solve this issue due to abilities of handling nonlinear features by kernel functions.Deep mining of log features indicating lithofacies still needs to be improved for kernel methods.Hence,this work employs deep neural networks to enhance the kernel principal component analysis(KPCA)method and proposes a deep kernel method(DKM)for lithofacies identification using well logs.DKM includes a feature extractor and a classifier.The feature extractor consists of a series of KPCA models arranged according to residual network structure.A gradient-free optimization method is introduced to automatically optimize parameters and structure in DKM,which can avoid complex tuning of parameters in models.To test the validation of the proposed DKM for lithofacies identification,an open-sourced dataset with seven con-ventional logs(GR,CAL,AC,DEN,CNL,LLD,and LLS)and lithofacies labels from the Daniudi Gas Field in China is used.There are eight lithofacies,namely clastic rocks(pebbly,coarse,medium,and fine sand-stone,siltstone,mudstone),coal,and carbonate rocks.The comparisons between DKM and three commonly used kernel methods(KFD,SVM,MSVM)show that(1)DKM(85.7%)outperforms SVM(77%),KFD(79.5%),and MSVM(82.8%)in accuracy of lithofacies identification;(2)DKM is about twice faster than the multi-kernel method(MSVM)with good accuracy.The blind well test in Well D13 indicates that compared with the other three methods DKM improves about 24%in accuracy,35%in precision,41%in recall,and 40%in F1 score,respectively.In general,DKM is an effective method for complex lithofacies identification.This work also discussed the optimal structure and classifier for DKM.Experimental re-sults show that(m_(1),m_(2),O)is the optimal model structure and linear svM is the optimal classifier.(m_(1),m_(2),O)means there are m KPCAs,and then m2 residual units.A workflow to determine an optimal classifier in DKM for lithofacies identification is proposed,too.

Comparison of Kernel Entropy Component Analysis with Several Dimensionality Reduction Methods

Dimensionality reduction techniques play an important role in data mining. Kernel entropy component analysis( KECA) is a newly developed method for data transformation and dimensionality reduction. This paper conducted a comparative study of KECA with other five dimensionality reduction methods,principal component analysis( PCA),kernel PCA( KPCA),locally linear embedding( LLE),laplacian eigenmaps( LAE) and diffusion maps( DM). Three quality assessment criteria, local continuity meta-criterion( LCMC),trustworthiness and continuity measure(T&C),and mean relative rank error( MRRE) are applied as direct performance indexes to assess those dimensionality reduction methods. Moreover,the clustering accuracy is used as an indirect performance index to evaluate the quality of the representative data gotten by those methods. The comparisons are performed on six datasets and the results are analyzed by Friedman test with the corresponding post-hoc tests. The results indicate that KECA shows an excellent performance in both quality assessment criteria and clustering accuracy assessing.

Kernel Factor Analysis Algorithm with Varimax

Kernal factor analysis （KFA） with vafimax was proposed by using Mercer kernel function which can map the data in the original space to a high-dimensional feature space, and was compared with the kernel principle component analysis （KPCA）. The results show that the best error rate in handwritten digit recognition by kernel factor analysis with vadmax （4.2%） was superior to KPCA （4.4%）. The KFA with varimax could more accurately image handwritten digit recognition.

Application of Particle Swarm Optimization to Fault Condition Recognition Based on Kernel Principal Component Analysis

Panicle swarm optimization （PSO） is an optimization algorithm based on the swarm intelligent principle. In this paper the modified PSO is applied to a kernel principal component analysis （ KPCA ） for an optimal kernel function parameter. We first comprehensively considered within-class scatter and between-class scatter of the sample features. Then, the fitness function of an optimized kernel function parameter is constructed, and the particle swarm optimization algorithm with adaptive acceleration （CPSO） is applied to optimizing it. It is used for gearbox condi- tion recognition, and the result is compared with the recognized results based on principal component analysis （PCA）. The results show that KPCA optimized by CPSO can effectively recognize fault conditions of the gearbox by reducing bind set-up of the kernel function parameter, and its results of fault recognition outperform those of PCA. We draw the conclusion that KPCA based on CPSO has an advantage in nonlinear feature extraction of mechanical failure, and is helpful for fault condition recognition of complicated machines.

Multi-state Information Dimension Reduction Based on Particle Swarm Optimization-Kernel Independent Component Analysis

The precision of the kernel independent component analysis( KICA) algorithm depends on the type and parameter values of kernel function. Therefore,it's of great significance to study the choice method of KICA's kernel parameters for improving its feature dimension reduction result. In this paper, a fitness function was established by use of the ideal of Fisher discrimination function firstly. Then the global optimal solution of fitness function was searched by particle swarm optimization( PSO) algorithm and a multi-state information dimension reduction algorithm based on PSO-KICA was established. Finally,the validity of this algorithm to enhance the precision of feature dimension reduction has been proven.

Nonlinear Statistical Process Monitoring Based on Control Charts with Memory Effect and Kernel Independent Component Analysis

A novel nonlinear combination process monitoring method was proposed based on techniques with memo- ry effect （multivariate exponentially weighted moving average （MEWMA）） and kernel independent component analysis （KICA）. The method was developed for dealing with nonlinear issues and detecting small or moderate drifts in one or more process variables with autocorrelation. MEWMA charts use additional information from the past history of the process for keeping the memory effect of the process behavior trend. KICA is a recently devel- oped statistical technique for revealing hidden, nonlinear statistically independent factors that underlie sets of mea- surements and it is a two-phase algorithm., whitened kernel principal component analysis （KPCA） plus indepen- dent component analysis （ICA）. The application to the fluid catalytic cracking unit （FCCU） simulated process in- dicates that the proposed combined method based on MEWMA and KICA can effectively capture the nonlinear rela- tionship and detect small drifts in process variables. Its performance significantly outperforms monitoring method based on ICA, MEWMA-ICA and KICA, especially for lonu-term performance deterioration.

Theoretical convergence analysis of complex Gaussian kernel LMS algorithm

With the vigorous expansion of nonlinear adaptive filtering with real-valued kernel functions,its counterpart complex kernel adaptive filtering algorithms were also sequentially proposed to solve the complex-valued nonlinear problems arising in almost all real-world applications.This paper firstly presents two schemes of the complex Gaussian kernel-based adaptive filtering algorithms to illustrate their respective characteristics.Then the theoretical convergence behavior of the complex Gaussian kernel least mean square（LMS） algorithm is studied by using the fixed dictionary strategy.The simulation results demonstrate that the theoretical curves predicted by the derived analytical models consistently coincide with the Monte Carlo simulation results in both transient and steady-state stages for two introduced complex Gaussian kernel LMS algonthms using non-circular complex data.The analytical models are able to be regard as a theoretical tool evaluating ability and allow to compare with mean square error（MSE） performance among of complex kernel LMS（KLMS） methods according to the specified kernel bandwidth and the length of dictionary.

Classification of Service Functions in the Life Circle of Beijing Subway Line 7

As a key node for the surrounding area,metro stations are closely connected with the surrounding urban space and develop cooperatively.Different types of metro stations have differences in land use and functional positioning.This paper mainly used the methods of Tyson polygon,kernel density analysis and correlation analysis,based on POI data,to classify the stations of Beijing Metro Line 7.This paper made a detailed analysis of commercial subway stations,and analyzed the distribution characteristics of commercial metro stations on Line 7.

Statistical Monitoring of Chemical Processes Based on Sensitive Kernel Principal Components

The kernel principal component analysis （KPCA） method employs the first several kernel principal components （KPCs）, which indicate the most variance information of normal observations for process monitoring, but may not reflect the fault information. In this study, sensitive kernel principal component analysis （SKPCA） is proposed to improve process monitoring performance, i.e., to deal with the discordance of T2 statistic and squared prediction error SVE statistic and reduce missed detection rates. T2 statistic can be used to measure the variation di rectly along each KPC and analyze the detection performance as well as capture the most useful information in a process. With the calculation of the change rate of T2 statistic along each KPC, SKPCA selects the sensitive kernel principal components for process monitoring. A simulated simple system and Tennessee Eastman process are employed to demonstrate the efficiency of SKPCA on online monitoring. The results indicate that the monitoring performance is improved significantly.

Numerical study of resting-state fMRI based on kernel ICA

In order to facilitate the extraction of the default mode network（DMN）, reduce the data complexity of the functional magnetic resonance imaging （fMRI）and overcome the restriction of the linearity of the mixing process encountered with the independent component analysis（ICA）, a framework of dimensionality reduction and nonlinear transformation is proposed. First, the principal component analysis（PCA）is applied to reduce the time dimension 153 594×128 of the fMRI data to 153 594×5 for simplifying complexity computation and obtaining 95% of the information. Secondly, a new kernel-based nonlinear ICA method referred as the kernel ICA（KICA）based on the Gaussian kernel is introduced to analyze the resting-state fMRI data and extract the DMN. Experimental results show that the KICA provides a better performance for the resting-state fMRI data analysis compared with the classical ICA. Furthermore, the DMN is accurately extracted and the noise is reduced.

Multivariate time delay analysis based local KPCA fault prognosis approach for nonlinear processes

Currently, some fault prognosis technology occasionally has relatively unsatisfied performance especially for in- cipient faults in nonlinear processes duo to their large time delay and complex internal connection. To overcome this deficiency, multivariate time delay analysis is incorporated into the high sensitive local kernel principal component analysis. In this approach, mutual information estimation and Bayesian information criterion （BIC） are separately used to acquire the correlation degree and time delay of the process variables. Moreover, in order to achieve prediction, time series prediction by back propagation （BP） network is applied whose input is multivar- iate correlated time series other than the original time series. Then the multivariate time delayed series and future values obtained by time series prediction are combined to construct the input of local kernel principal component analysis （LKPCA） model for incipient fault prognosis. The new method has been exemplified in a sim- ple nonlinear process and the complicated Tennessee Eastman （TE） benchmark process. The results indicate that the new method has suoerioritv in the fault prognosis sensitivity over other traditional fault prognosis methods.

Air Traffic Operation Complexity Analysis Based on Metrics System

In order to quantitatively analyze air traffic operation complexity,multidimensional metrics were selected based on the operational characteristics of traffic flow.The kernel principal component analysis method was utilized to reduce the dimensionality of metrics,therefore to extract crucial information in the metrics.The hierarchical clustering method was used to analyze the complexity of different airspace.Fourteen sectors of Guangzhou Area Control Center were taken as samples.The operation complexity of traffic situation in each sector was calculated based on real flight radar data.Clustering analysis verified the feasibility and rationality of the method,and provided a reference for airspace operation and management.

Application of Kernel GDA to Performance Monitoring and Fault Diagnosis for Rotating Machinery

Faults in rotating machine are difficult to detect and identify,especially when the system is complex and nonlinear.In order to solve this problem,a novel performance monitoring and fault diagnosis method based on kernel generalized discriminant analysis(kernel GDA,KGDA)was proposed.Through KGDA,the data were mapped from the original space to the high-dimensional feature space.Then the statistic distance between normal data and test data was constructed to detect whether a fault was occurring.If a fault had occurred,similar analysis was used to identify the type of faults.The effectiveness of the proposed method was evaluated by simulation results of vibration signal fault dataset in the rotating machinery,which was scalable to different rotating machinery.

Modeling and monitoring of nonlinear multi-mode processes based on similarity measure-KPCA

A new modeling and monitoring approach for multi-mode processes is proposed.The method of similarity measure(SM) and kernel principal component analysis(KPCA) are integrated to construct SM-KPCA monitoring scheme,where SM method serves as the separation of common subspace and specific subspace.Compared with the traditional methods,the main contributions of this work are:1) SM consisted of two measures of distance and angle to accommodate process characters.The different monitoring effect involves putting on the different weight,which would simplify the monitoring model structure and enhance its reliability and robustness.2) The proposed method can be used to find faults by the common space and judge which mode the fault belongs to by the specific subspace.Results of algorithm analysis and fault detection experiments indicate the validity and practicability of the presented method.

Adaptive WNN aerodynamic modeling based on subset KPCA feature extraction

In order to accurately describe the dynamic characteristics of flight vehicles through aerodynamic modeling, an adaptive wavelet neural network (AWNN) aerodynamic modeling method is proposed, based on subset kernel principal components analysis (SKPCA) feature extraction. Firstly, by fuzzy C-means clustering, some samples are selected from the training sample set to constitute a sample subset. Then, the obtained samples subset is used to execute SKPCA for extracting basic features of the training samples. Finally, using the extracted basic features, the AWNN aerodynamic model is established. The experimental results show that, in 50 times repetitive modeling, the modeling ability of the method proposed is better than that of other six methods. It only needs about half the modeling time of KPCA-AWNN under a close prediction accuracy, and can easily determine the model parameters. This enables it to be effective and feasible to construct the aerodynamic modeling for flight vehicles.