Convergence analysis for complementary-label learning with kernel ridge regression

Complementary-label learning(CLL)aims at finding a classifier via samples with complementary labels.Such data is considered to contain less information than ordinary-label samples.The transition matrix between the true label and the complementary label,and some loss functions have been developed to handle this problem.In this paper,we show that CLL can be transformed into ordinary classification under some mild conditions,which indicates that the complementary labels can supply enough information in most cases.As an example,an extensive misclassification error analysis was performed for the Kernel Ridge Regression(KRR)method applied to multiple complementary-label learning(MCLL),which demonstrates its superior performance compared to existing approaches.

AHermitian C^(2) Differential Reproducing Kernel Interpolation Meshless Method for the 3D Microstructure-Dependent Static Flexural Analysis of Simply Supported and Functionally Graded Microplates

This work develops a Hermitian C^(2) differential reproducing kernel interpolation meshless(DRKIM)method within the consistent couple stress theory(CCST)framework to study the three-dimensional(3D)microstructuredependent static flexural behavior of a functionally graded(FG)microplate subjected to mechanical loads and placed under full simple supports.In the formulation,we select the transverse stress and displacement components and their first-and second-order derivatives as primary variables.Then,we set up the differential reproducing conditions(DRCs)to obtain the shape functions of the Hermitian C^(2) differential reproducing kernel(DRK)interpolant’s derivatives without using direct differentiation.The interpolant’s shape function is combined with a primitive function that possesses Kronecker delta properties and an enrichment function that constituents DRCs.As a result,the primary variables and their first-and second-order derivatives satisfy the nodal interpolation properties.Subsequently,incorporating ourHermitianC^(2)DRKinterpolant intothe strong formof the3DCCST,we develop a DRKIM method to analyze the FG microplate’s 3D microstructure-dependent static flexural behavior.The Hermitian C^(2) DRKIM method is confirmed to be accurate and fast in its convergence rate by comparing the solutions it produces with the relevant 3D solutions available in the literature.Finally,the impact of essential factors on the transverse stresses,in-plane stresses,displacements,and couple stresses that are induced in the loaded microplate is examined.These factors include the length-to-thickness ratio,the material length-scale parameter,and the inhomogeneity index,which appear to be significant.

h-ADAPTIVITY ANALYSIS BASED ON MULTIPLE SCALE REPRODUCING KERNEL PARTICLE METHOD

An h-adaptivity analysis scheme based on multiple scale reproducing kernel particle method was proposed, and two node refinement strategies were constructed using searching-neighbor-nodes(SNN) and local-Delaunay-triangulation(LDT) tech-niques, which were suitable and effective for h-adaptivity analysis on 2-D problems with the regular or irregular distribution of the nodes. The results of multiresolution and h-adaptivity analyses on 2-D linear elastostatics and bending plate problems demonstrate that the improper high-gradient indicator will reduce the convergence property of the h-adaptivity analysis, and that the efficiency of the LDT node refinement strategy is better than SNN, and that the presented h-adaptivity analysis scheme is provided with the validity, stability and good convergence property.

Kernel principal component analysis network for image classification

In order to classify nonlinear features with a linear classifier and improve the classification accuracy, a deep learning network named kernel principal component analysis network（ KPCANet） is proposed. First, the data is mapped into a higher-dimensional space with kernel principal component analysis to make the data linearly separable. Then a two-layer KPCANet is built to obtain the principal components of the image. Finally, the principal components are classified with a linear classifier. Experimental results showthat the proposed KPCANet is effective in face recognition, object recognition and handwritten digit recognition. It also outperforms principal component analysis network（ PCANet） generally. Besides, KPCANet is invariant to illumination and stable to occlusion and slight deformation.

NONLINEAR DATA RECONCILIATION METHOD BASED ON KERNEL PRINCIPAL COMPONENT ANALYSIS

In the industrial process situation, principal component analysis (PCA) is ageneral method in data reconciliation. However, PCA sometime is unfeasible to nonlinear featureanalysis and limited in application to nonlinear industrial process. Kernel PCA (KPCA) is extensionof PCA and can be used for nonlinear feature analysis. A nonlinear data reconciliation method basedon KPCA is proposed. The basic idea of this method is that firstly original data are mapped to highdimensional feature space by nonlinear function, and PCA is implemented in the feature space. Thennonlinear feature analysis is implemented and data are reconstructed by using the kernel. The datareconciliation method based on KPCA is applied to ternary distillation column. Simulation resultsshow that this method can filter the noise in measurements of nonlinear process and reconciliateddata can represent the true information of nonlinear process.

h-ADAPTIVITY ANALYSIS BASED ON MULTIPLE SCALE REPRODUCING KERNEL PARTICLE METHOD

An h-adaptivity analysis scheme based on multiple scale reproducing kernel particle method was proposed, and two node refinement strategies were constructed using searching-neighbor-nodes（SNN） and local-Delaunay-triangulation（LDT） techniques, which were suitable and effective for h-adaptivity analysis on 2-D problems with the regular or irregular distribution of the nodes. The results of multiresolution and h- adaptivity analyses on 2-D linear elastostatics and bending plate problems demonstrate that the improper high-gradient indicator will reduce the convergence property of the h- adaptivity analysis, and that the efficiency of the LDT node refinement strategy is better than SNN, and that the presented h-adaptivity analysis scheme is provided with the validity, stability and good convergence property.

A Quantized Kernel Least Mean Square Scheme with Entropy-Guided Learning for Intelligent Data Analysis

Quantized kernel least mean square(QKLMS) algorithm is an effective nonlinear adaptive online learning algorithm with good performance in constraining the growth of network size through the use of quantization for input space. It can serve as a powerful tool to perform complex computing for network service and application. With the purpose of compressing the input to further improve learning performance, this article proposes a novel QKLMS with entropy-guided learning, called EQ-KLMS. Under the consecutive square entropy learning framework, the basic idea of entropy-guided learning technique is to measure the uncertainty of the input vectors used for QKLMS, and delete those data with larger uncertainty, which are insignificant or easy to cause learning errors. Then, the dataset is compressed. Consequently, by using square entropy, the learning performance of proposed EQ-KLMS is improved with high precision and low computational cost. The proposed EQ-KLMS is validated using a weather-related dataset, and the results demonstrate the desirable performance of our scheme.

FUZZY PRINCIPAL COMPONENT ANALYSIS AND ITS KERNEL-BASED MODEL

Principal Component Analysis(PCA)is one of the most important feature extraction methods,and Kernel Principal Component Analysis(KPCA)is a nonlinear extension of PCA based on kernel methods.In real world,each input data may not be fully assigned to one class and it may partially belong to other classes.Based on the theory of fuzzy sets,this paper presents Fuzzy Principal Component Analysis(FPCA)and its nonlinear extension model,i.e.,Kernel-based Fuzzy Principal Component Analysis(KFPCA).The experimental results indicate that the proposed algorithms have good performances.

A Kernel Time Structure Independent Component Analysis Method for Nonlinear Process Monitoring

Kernel independent component analysis(KICA) is a newly emerging nonlinear process monitoring method,which can extract mutually independent latent variables called independent components(ICs) from process variables. However, when more than one IC have Gaussian distribution, it cannot extract the IC feature effectively and thus its monitoring performance will be degraded drastically. To solve such a problem, a kernel time structure independent component analysis(KTSICA) method is proposed for monitoring nonlinear process in this paper. The original process data are mapped into a feature space nonlinearly and then the whitened data are calculated in the feature space by the kernel trick. Subsequently, a time structure independent component analysis algorithm, which has no requirement for the distribution of ICs, is proposed to extract the IC feature.Finally, two monitoring statistics are built to detect process faults. When some fault is detected, a nonlinear fault identification method is developed to identify fault variables based on sensitivity analysis. The proposed monitoring method is applied in the Tennessee Eastman benchmark process. Applications demonstrate the superiority of KTSICA over KICA.

Kernel Generalization of Multi-Rate Probabilistic Principal Component Analysis for Fault Detection in Nonlinear Process

In practical process industries,a variety of online and offline sensors and measuring instruments have been used for process control and monitoring purposes,which indicates that the measurements coming from different sources are collected at different sampling rates.To build a complete process monitoring strategy,all these multi-rate measurements should be considered for data-based modeling and monitoring.In this paper,a novel kernel multi-rate probabilistic principal component analysis(K-MPPCA)model is proposed to extract the nonlinear correlations among different sampling rates.In the proposed model,the model parameters are calibrated using the kernel trick and the expectation-maximum(EM)algorithm.Also,the corresponding fault detection methods based on the nonlinear features are developed.Finally,a simulated nonlinear case and an actual pre-decarburization unit in the ammonia synthesis process are tested to demonstrate the efficiency of the proposed method.

Feature Fusion Multi-View Hashing Based on Random Kernel Canonical Correlation Analysis

Hashing technology has the advantages of reducing data storage and improving the efficiency of the learning system,making it more and more widely used in image retrieval.Multi-view data describes image information more comprehensively than traditional methods using a single-view.How to use hashing to combine multi-view data for image retrieval is still a challenge.In this paper,a multi-view fusion hashing method based on RKCCA(Random Kernel Canonical Correlation Analysis)is proposed.In order to describe image content more accurately,we use deep learning dense convolutional network feature DenseNet to construct multi-view by combining GIST feature or BoW_SIFT(Bag-of-Words model+SIFT feature)feature.This algorithm uses RKCCA method to fuse multi-view features to construct association features and apply them to image retrieval.The algorithm generates binary hash code with minimal distortion error by designing quantization regularization terms.A large number of experiments on benchmark datasets show that this method is superior to other multi-view hashing methods.

Cobalt crust recognition based on kernel Fisher discriminant analysis and genetic algorithm in reverberation environment

Recognition of substrates in cobalt crust mining areas can improve mining efficiency.Aiming at the problem of unsatisfactory performance of using single feature to recognize the seabed material of the cobalt crust mining area,a method based on multiple-feature sets is proposed.Features of the target echoes are extracted by linear prediction method and wavelet analysis methods,and the linear prediction coefficient and linear prediction cepstrum coefficient are also extracted.Meanwhile,the characteristic matrices of modulus maxima,sub-band energy and multi-resolution singular spectrum entropy are obtained,respectively.The resulting features are subsequently compressed by kernel Fisher discriminant analysis(KFDA),the output features are selected using genetic algorithm(GA)to obtain optimal feature subsets,and recognition results of classifier are chosen as genetic fitness function.The advantages of this method are that it can describe the signal features more comprehensively and select the favorable features and remove the redundant features to the greatest extent.The experimental results show the better performance of the proposed method in comparison with only using KFDA or GA.

Kernel Model Applied in Kernel Direct Discriminant Analysis for the Recognition of Face with Nonlinear Variations

A kernel-based discriminant analysis method called kernel direct discriminant analysis is employed, which combines the merit of direct linear discriminant analysis with that of kernel trick. In order to demonstrate its better robustness to the complex and nonlinear variations of real face images, such as illumination, facial expression, scale and pose variations, experiments are carried out on the Olivetti Research Laboratory, Yale and self-built face databases. The results indicate that in contrast to kernel principal component analysis and kernel linear discriminant analysis, the method can achieve lower （7%） error rate using only a very small set of features. Furthermore, a new corrected kernel model is proposed to improve the recognition performance. Experimental results confirm its superiority （1% in terms of recognition rate） to other polynomial kernel models.

Attractive Multistep Reproducing Kernel Approach for Solving Stiffness Differential Systems of Ordinary Differential Equations and Some Error Analysis

In this paper,an efficient multi-step scheme is presented based on reproducing kernel Hilbert space(RKHS)theory for solving ordinary stiff differential systems.The solution methodology depends on reproducing kernel functions to obtain analytic solutions in a uniform formfor a rapidly convergent series in the posed Sobolev space.Using the Gram-Schmidt orthogonality process,complete orthogonal essential functions are obtained in a compact field to encompass Fourier series expansion with the help of kernel properties reproduction.Consequently,by applying the standard RKHS method to each subinterval,approximate solutions that converge uniformly to the exact solutions are obtained.For this purpose,several numerical examples are tested to show proposed algorithm’s superiority,simplicity,and efficiency.The gained results indicate that themulti-step RKHSmethod is suitable for solving linear and nonlinear stiffness systems over an extensive duration and giving highly accurate outcomes.

Face Recognition Using Kernel Discriminant Analysis

Linear discrimiant analysis (LDA) has been used in face recognition. But it is difficult to handle the high nonlinear problems, such as changes of large viewpoint and illumination. In order to overcome these problems, kernel discriminant analysis for face recognition is presented. This approach adopts the kernel functions to replace the dot products of nonlinear mapping in the high dimensional feature space, and then the nonlinear problem can be solved in the input space conveniently without explicit mapping. Two face databases are given.

Robust Classification through a Nonparametric Kernel Discriminant Analysis

The problem of classification in situations where the assumption of normality in the data is violated, and there are non-linear clustered structures in the dataset is addressed. A robust nonparametric kernel discriminant classification function, which is able to address this challenge, has been developed and the misclassification rates computed for various bandwidth matrices. A comparison with existing parametric classification functions such as the linear discriminant and quadratic discriminant is conducted to evaluate the performance of this classification function using simulated datasets. The results presented in this paper show good performance in terms of misclassification rates for the kernel discriminant classifier when the correct bandwidth is selected as compared to other identified existing classifiers. In this regard, the study recommends the use of the proposed kernel discriminant classification rule when one wishes to classify units into one of several categories or population groups where parametric classifiers might not be applicable.

Stability and dispersion analysis of reproducing kernel collocation method for transient dynamics

A reproducing kernel collocation method based on strong formulation is introduced for transient dynamics. To study the stability property of this method, an algorithm based on the von Neumann hypothesis is proposed to predict the critical time step. A numerical test is conducted to validate the algorithm. The numerical critical time step and the predicted critical time step are in good agreement. The results are compared with those obtained based on the radial basis collocation method, and they axe in good agreement. Several important conclusions for choosing a proper support size of the reproducing kernel shape function are given to improve the stability condition.

Comparison of Kernel Entropy Component Analysis with Several Dimensionality Reduction Methods

Dimensionality reduction techniques play an important role in data mining. Kernel entropy component analysis( KECA) is a newly developed method for data transformation and dimensionality reduction. This paper conducted a comparative study of KECA with other five dimensionality reduction methods,principal component analysis( PCA),kernel PCA( KPCA),locally linear embedding( LLE),laplacian eigenmaps( LAE) and diffusion maps( DM). Three quality assessment criteria, local continuity meta-criterion( LCMC),trustworthiness and continuity measure(T&C),and mean relative rank error( MRRE) are applied as direct performance indexes to assess those dimensionality reduction methods. Moreover,the clustering accuracy is used as an indirect performance index to evaluate the quality of the representative data gotten by those methods. The comparisons are performed on six datasets and the results are analyzed by Friedman test with the corresponding post-hoc tests. The results indicate that KECA shows an excellent performance in both quality assessment criteria and clustering accuracy assessing.

Linear Discriminant Analysis and Kernel Vector Quantization for Mandarin Digits Recognition

Linear discriminant analysis and kernel vector quantization are integrated into vector quantization based speech recognition system for improving the recognition accuracy of Mandarin digits. These techniques increase the class separability and optimize the clustering procedure. Speaker-dependent (SD) and speaker-independent (SI) experiments are performed to evaluate the performance of the proposed method. The experiment results show that the proposed method is capable of reaching the word error rate of 3.76% in SD case and 6.60 % in SI case. Such a system can be suitable for being embedded in personal digital assistant(PDA), mobile phone and so on to perform voice controlling such as digit dialing, calculating, etc.