Kernel principal component analysis network for image classification

In order to classify nonlinear features with a linear classifier and improve the classification accuracy, a deep learning network named kernel principal component analysis network（ KPCANet） is proposed. First, the data is mapped into a higher-dimensional space with kernel principal component analysis to make the data linearly separable. Then a two-layer KPCANet is built to obtain the principal components of the image. Finally, the principal components are classified with a linear classifier. Experimental results showthat the proposed KPCANet is effective in face recognition, object recognition and handwritten digit recognition. It also outperforms principal component analysis network（ PCANet） generally. Besides, KPCANet is invariant to illumination and stable to occlusion and slight deformation.

Application of Particle Swarm Optimization to Fault Condition Recognition Based on Kernel Principal Component Analysis

Panicle swarm optimization （PSO） is an optimization algorithm based on the swarm intelligent principle. In this paper the modified PSO is applied to a kernel principal component analysis （ KPCA ） for an optimal kernel function parameter. We first comprehensively considered within-class scatter and between-class scatter of the sample features. Then, the fitness function of an optimized kernel function parameter is constructed, and the particle swarm optimization algorithm with adaptive acceleration （CPSO） is applied to optimizing it. It is used for gearbox condi- tion recognition, and the result is compared with the recognized results based on principal component analysis （PCA）. The results show that KPCA optimized by CPSO can effectively recognize fault conditions of the gearbox by reducing bind set-up of the kernel function parameter, and its results of fault recognition outperform those of PCA. We draw the conclusion that KPCA based on CPSO has an advantage in nonlinear feature extraction of mechanical failure, and is helpful for fault condition recognition of complicated machines.

Robust Recommendation Algorithm Based on Kernel Principal Component Analysis and Fuzzy C-means Clustering

The existing recommendation algorithms have lower robustness in facing of shilling attacks. Considering this problem, we present a robust recommendation algorithm based on kernel principal component analysis and fuzzy c-means clustering. Firstly, we use kernel principal component analysis method to reduce the dimensionality of the original rating matrix, which can extract the effective features of users and items. Then, according to the dimension-reduced rating matrix and the high correlation characteristic between attack profiles, we use fuzzy c-means clustering method to cluster user profiles, which can realize the effective separation of genuine profiles and attack profiles. Finally, we construct an indicator function based on the attack detection results to decrease the influence of attack profiles on the recommendation, and incorporate it into the matrix factorization technology to design the corresponding robust recommendation algorithm. Experiment results indicate that the proposed algorithm is superior to the existing methods in both recommendation accuracy and robustness.

Adaptive multiblock kernel principal component analysis for monitoring complex industrial processes

Multiblock kernel principal component analysis (MBKPCA) has been proposed to isolate the faults and avoid the high computation cost. However, MBKPCA is not available for dynamic processes. To solve this problem, recursive MBKPCA is proposed for monitoring large scale processes. In this paper, we present a new recursive MBKPCA (RMBKPCA) algorithm, where the adaptive technique is adopted for dynamic characteristics. The proposed algorithm reduces the high computation cost, and is suitable for online model updating in the feature space. The proposed algorithm was applied to an industrial process for adaptive monitoring and found to efficiently capture the time-varying and nonlinear relationship in the process variables.

Statistical Monitoring of Chemical Processes Based on Sensitive Kernel Principal Components

The kernel principal component analysis （KPCA） method employs the first several kernel principal components （KPCs）, which indicate the most variance information of normal observations for process monitoring, but may not reflect the fault information. In this study, sensitive kernel principal component analysis （SKPCA） is proposed to improve process monitoring performance, i.e., to deal with the discordance of T2 statistic and squared prediction error SVE statistic and reduce missed detection rates. T2 statistic can be used to measure the variation di rectly along each KPC and analyze the detection performance as well as capture the most useful information in a process. With the calculation of the change rate of T2 statistic along each KPC, SKPCA selects the sensitive kernel principal components for process monitoring. A simulated simple system and Tennessee Eastman process are employed to demonstrate the efficiency of SKPCA on online monitoring. The results indicate that the monitoring performance is improved significantly.

Comparison of Kernel Entropy Component Analysis with Several Dimensionality Reduction Methods

Dimensionality reduction techniques play an important role in data mining. Kernel entropy component analysis( KECA) is a newly developed method for data transformation and dimensionality reduction. This paper conducted a comparative study of KECA with other five dimensionality reduction methods,principal component analysis( PCA),kernel PCA( KPCA),locally linear embedding( LLE),laplacian eigenmaps( LAE) and diffusion maps( DM). Three quality assessment criteria, local continuity meta-criterion( LCMC),trustworthiness and continuity measure(T&C),and mean relative rank error( MRRE) are applied as direct performance indexes to assess those dimensionality reduction methods. Moreover,the clustering accuracy is used as an indirect performance index to evaluate the quality of the representative data gotten by those methods. The comparisons are performed on six datasets and the results are analyzed by Friedman test with the corresponding post-hoc tests. The results indicate that KECA shows an excellent performance in both quality assessment criteria and clustering accuracy assessing.

A deep kernel method for lithofacies identification using conventional well logs

How to fit a properly nonlinear classification model from conventional well logs to lithofacies is a key problem for machine learning methods.Kernel methods(e.g.,KFD,SVM,MSVM)are effective attempts to solve this issue due to abilities of handling nonlinear features by kernel functions.Deep mining of log features indicating lithofacies still needs to be improved for kernel methods.Hence,this work employs deep neural networks to enhance the kernel principal component analysis(KPCA)method and proposes a deep kernel method(DKM)for lithofacies identification using well logs.DKM includes a feature extractor and a classifier.The feature extractor consists of a series of KPCA models arranged according to residual network structure.A gradient-free optimization method is introduced to automatically optimize parameters and structure in DKM,which can avoid complex tuning of parameters in models.To test the validation of the proposed DKM for lithofacies identification,an open-sourced dataset with seven con-ventional logs(GR,CAL,AC,DEN,CNL,LLD,and LLS)and lithofacies labels from the Daniudi Gas Field in China is used.There are eight lithofacies,namely clastic rocks(pebbly,coarse,medium,and fine sand-stone,siltstone,mudstone),coal,and carbonate rocks.The comparisons between DKM and three commonly used kernel methods(KFD,SVM,MSVM)show that(1)DKM(85.7%)outperforms SVM(77%),KFD(79.5%),and MSVM(82.8%)in accuracy of lithofacies identification;(2)DKM is about twice faster than the multi-kernel method(MSVM)with good accuracy.The blind well test in Well D13 indicates that compared with the other three methods DKM improves about 24%in accuracy,35%in precision,41%in recall,and 40%in F1 score,respectively.In general,DKM is an effective method for complex lithofacies identification.This work also discussed the optimal structure and classifier for DKM.Experimental re-sults show that(m_(1),m_(2),O)is the optimal model structure and linear svM is the optimal classifier.(m_(1),m_(2),O)means there are m KPCAs,and then m2 residual units.A workflow to determine an optimal classifier in DKM for lithofacies identification is proposed,too.

Kernel Factor Analysis Algorithm with Varimax

Kernal factor analysis （KFA） with vafimax was proposed by using Mercer kernel function which can map the data in the original space to a high-dimensional feature space, and was compared with the kernel principle component analysis （KPCA）. The results show that the best error rate in handwritten digit recognition by kernel factor analysis with vadmax （4.2%） was superior to KPCA （4.4%）. The KFA with varimax could more accurately image handwritten digit recognition.

Multivariate time delay analysis based local KPCA fault prognosis approach for nonlinear processes

Currently, some fault prognosis technology occasionally has relatively unsatisfied performance especially for in- cipient faults in nonlinear processes duo to their large time delay and complex internal connection. To overcome this deficiency, multivariate time delay analysis is incorporated into the high sensitive local kernel principal component analysis. In this approach, mutual information estimation and Bayesian information criterion （BIC） are separately used to acquire the correlation degree and time delay of the process variables. Moreover, in order to achieve prediction, time series prediction by back propagation （BP） network is applied whose input is multivar- iate correlated time series other than the original time series. Then the multivariate time delayed series and future values obtained by time series prediction are combined to construct the input of local kernel principal component analysis （LKPCA） model for incipient fault prognosis. The new method has been exemplified in a sim- ple nonlinear process and the complicated Tennessee Eastman （TE） benchmark process. The results indicate that the new method has suoerioritv in the fault prognosis sensitivity over other traditional fault prognosis methods.

Cycle temporal algorithm-based multivariate statistical methods for fault diagnosis in chemical processes

Multivariate statistical process monitoring methods are often used in chemical process fault diagnosis.In this article,(I)the cycle temporal algorithm(CTA)combined with the dynamic kernel principal component analysis(DKPCA)and the multiway dynamic kernel principal component analysis(MDKPCA)fault detection algorithms are proposed,which are used for continuous and batch process fault detections,respectively.In addition,(II)a fault variable identification model based on reconstructed-based contribution(RBC)model that paves the way for determining the cause of the fault are proposed.The proposed fault diagnosis model was applied to Tennessee Eastman(TE)process and penicillin fermentation process for fault diagnosis.And compare with other fault diagnosis methods.The results show that the proposed method has better detection effects than other methods.Finally,the reconstruction-based contribution(RBC)model method is used to accurately locate the root cause of the fault and determine the fault path.

Modeling and monitoring of nonlinear multi-mode processes based on similarity measure-KPCA

A new modeling and monitoring approach for multi-mode processes is proposed.The method of similarity measure(SM) and kernel principal component analysis(KPCA) are integrated to construct SM-KPCA monitoring scheme,where SM method serves as the separation of common subspace and specific subspace.Compared with the traditional methods,the main contributions of this work are:1) SM consisted of two measures of distance and angle to accommodate process characters.The different monitoring effect involves putting on the different weight,which would simplify the monitoring model structure and enhance its reliability and robustness.2) The proposed method can be used to find faults by the common space and judge which mode the fault belongs to by the specific subspace.Results of algorithm analysis and fault detection experiments indicate the validity and practicability of the presented method.

Forest fire smoke recognition based on convolutional neural network

Traditional fire smoke detection methods mostly rely on manual algorithm extraction and sensor detection;however,these methods are slow and expensive to achieve discrimination.We proposed an improved convolutional neural network(CNN)to achieve fast analysis.The improved CNN can be used to liberate manpower.The network does not require complicated manual feature extraction to identify forest fire smoke.First,to alleviate the computational pressure and speed up the discrimination efficiency,kernel principal component analysis was performed on the experimental data set.To improve the robustness of the CNN and to avoid overfitting,optimization strategies were applied in multi-convolution kernels and batch normalization to improve loss functions.The experimental analysis shows that the CNN proposed in this study can learn the feature information automatically for smoke images in the early stages of fire automatically with a high recognition rate.As a result,the improved CNN enriches the theory of smoke discrimination in the early stages of a forest fire.

Analog circuit intelligent fault diagnosis based on GKPCA and multi-class SVM approach

Analog circuits fault diagnosis is essential for guaranteeing the reliability and maintainability of electronic systems. In this paper, a novel analog circuit fault diagnosis approach is proposed based on greedy kernel principal component analysis (KPCA) and one-against-all support vector machine (OAASVM). In order to obtain a successful SVM-based fault classifier, eliminating noise and extracting fault features are very important. Due to the better performance of nonlinear fault features extraction and noise elimination as compared with PCA, KPCA is adopted in the proposed approach. However, when we adopt KPCA to extract fault features of analog circuit, a drawback of KPCA is that the storage required for the kernel matrix grows quadratically, and the computational cost for eigenvector of the kernel matrix grows linearly with the number of training samples. Therefore, GKPCA, which can approximate KPCA with small representation error, is introduced to enhance computational efficiency. Based on the statistical learning theory and the empirical risk minimization principle, SVM has advantages of better classification accuracy and generalization performance. The extracted fault features are then used as the inputs of OAASVM to solve fault diagnosis problem. The effectiveness of the proposed approach is verified by the experimental results.

Fault diagnosis method of track circuit based on KPCA-SAE

At present,ZPW-2000 track circuit fault diagnosis is artificially analyzed and monitored.Its discrimination method not only is low efficient and takes a long period,but also requires highly experienced personnel to analyze the data.Therefore,we introduce kernel principal component analysis and stacked auto-encoder network(KPCA-SAD)into the fault diagnosis of ZPW-2000 track circuit.According to the working principle and fault characteristics of track circuit,a fault diagnosis model of KPCA-SAE network is established.The relevant parameters of key components recorded in the data collected by field staff are used as the fault feature parameters.The KPCA method is used to reduce the dimension and noise of fault document matrix to avoid information redundancy.The SAE network is trained by the processed fault data.The model parameters are optimized overall by using back propagation(BP)algorithm.The KPCA-SAE model is simulated in Matlab platform and is finally proved to be effective and feasible.Compared with the traditional method of artificially analyzing fault data and other intelligent algorithms,the KPCA-SAE based classifier has higher fault identification accuracy.

Forecasting Model of Photovoltaic Power Based on KPCA-MCS-DCNN

Accurate photovoltaic(PV)power prediction can effectively help the power sector to make rational energy planning and dispatching decisions,promote PV consumption,make full use of renewable energy and alleviate energy problems.To address this research objective,this paper proposes a prediction model based on kernel principal component analysis(KPCA),modified cuckoo search algorithm(MCS)and deep convolutional neural networks(DCNN).Firstly,KPCA is utilized to reduce the dimension of the feature,which aims to reduce the redundant input vectors.Then using MCS to optimize the parameters of DCNN.Finally,the photovoltaic power forecasting method of KPCA-MCS-DCNN is established.In order to verify the prediction performance of the proposed model,this paper selects a photovoltaic power station in China for example analysis.The results show that the new hybrid KPCA-MCS-DCNN model has higher prediction accuracy and better robustness.

Comparison between PCA and KPCA methods in comprehensive evaluation of robotic kinematic dexterity

Due to the correlation and diversity of robotic kinematic dexterity indexes, the principal component analysis （PCA） and kernel principal component analysis （KPCA） based on linear dimension reduction and nonlinear dimension reduction principle could be respectively introduced into comprehensive kinematic dexterity performance evaluation of space 3R robot of different tasks. By comparing different dimension reduction effects, the KPCA method could deal more effectively with the nonlinear relationship among different single kinematic dexterity indexes, and its calculation result is more reasonable for containing more comprehensive information. KPCA＇ s calculation provides scientific basis for optimum order of robotic tasks, and furthermore a new optimization method for robotic task selection is proposed based on various performance indexes.

Method and practice of deep favorable shale reservoirs prediction based on machine learning

A set of methods for predicting the favorable reservoir of deep shale gas based on machine learning is proposed through research of parameter correlation feature analysis principle, intelligent prediction method based on convolution neural network(CNN), and integrated fusion characterization method based on kernel principal component analysis(KPCA) nonlinear dimension reduction principle.(1) High-dimensional correlation characteristics of core and logging data are analyzed based on the Pearson correlation coefficient.(2) The nonlinear dimension reduction method of KPCA is used to characterize complex high-dimensional data to efficiently and accurately understand the core and logging response laws to favorable reservoirs.(3) CNN and logging data are used to train and verify the model similar to the underground reservoir.(4) CNN and seismic data are used to intelligently predict favorable reservoir parameters such as organic carbon content, gas content, brittleness and in-situ stress to effectively solve the problem of nonlinear and complex feature extraction in reservoir prediction.(5) KPCA is used to eliminate complex redundant information, mine big data characteristics of favorable reservoirs, and integrate and characterize various parameters to comprehensively evaluate reservoirs. This method has been used to predict the spatial distribution of favorable shale reservoirs in the Ordovician Wufeng Formation to the Silurian Longmaxi Formation of the Weirong shale gas field in the Sichuan Basin, SW China. The predicted results are highly consistent with the actual core, logging and productivity data, proving that this method can provide effective support for the exploration and development of deep shale gas.

Multi-view recognition of fruit packing boxes based on features clustering angle

In order to realize the intelligent mechanization of the last process of the fruit industry chains,the identification of fruit packing boxes is researched.A multi-view database is established to describe the omnidirectional attitudes of the fruit packing boxes.In order to reduce the data redundancy caused by multi-view acquisition,a new binary multi-view kernel principal component analysis network(BMKPCANet) is built,and a multi-view recognition method of fruit packing boxes is proposed based on the BMKPCANet and support vector machine(SVM).The experimental results show that the recognition accuracy of proposed BMKPCANet is 12.82% higher than PCANet and3.51% higher than KPCANet on average.The time consumption of proposed BMKPCANet is 7.74%lower than PCANet and 29.01% lower than KPCANet on average.This work has laid a theoretical foundation for multi-view recognition of 3 D objects and has a good practical application value.

Adaptive WNN aerodynamic modeling based on subset KPCA feature extraction

In order to accurately describe the dynamic characteristics of flight vehicles through aerodynamic modeling, an adaptive wavelet neural network (AWNN) aerodynamic modeling method is proposed, based on subset kernel principal components analysis (SKPCA) feature extraction. Firstly, by fuzzy C-means clustering, some samples are selected from the training sample set to constitute a sample subset. Then, the obtained samples subset is used to execute SKPCA for extracting basic features of the training samples. Finally, using the extracted basic features, the AWNN aerodynamic model is established. The experimental results show that, in 50 times repetitive modeling, the modeling ability of the method proposed is better than that of other six methods. It only needs about half the modeling time of KPCA-AWNN under a close prediction accuracy, and can easily determine the model parameters. This enables it to be effective and feasible to construct the aerodynamic modeling for flight vehicles.

GLCM Based Extraction of Flame Image Texture Features and KPCA-GLVQ Recognition Method for Rotary Kiln Combustion Working Conditions

According to the pulverized coal combustion flame image texture features of the rotary-kiln oxide pellets sintering process,a combustion working condition recognition method based on the generalized learning vector(GLVQ) neural network is proposed.Firstly,the numerical flame image is analyzed to extract texture features,such as energy,entropy and inertia,based on grey-level co-occurrence matrix(GLCM) to provide qualitative information on the changes in the visual appearance of the flame.Then the kernel principal component analysis(KPCA) method is adopted to deduct the input vector with high dimensionality so as to reduce the GLVQ target dimension and network scale greatly.Finally,the GLVQ neural network is trained by using the normalized texture feature data.The test results show that the proposed KPCA-GLVQ classifer has an excellent performance on training speed and correct recognition rate,and it meets the requirement for real-time combustion working condition recognition for the rotary kiln process.