Stability of Fredholm Integral Equation of the First Kind in Reproducing Kernel Space

It is well known that the problem on the stability of the solutions for Fredholm integral equation of the first kind is an ill-posed problem in C[a, b] or L2 [a, b]. In this paper, the representation of the solution for Fredholm integral equation of the first kind is given if it has a unique solution. The stability of the solution is proved in the reproducing kernel space, namely, the measurement errors of the experimental data cannot result in unbounded errors of the true solution. The computation of approximate solution is also stable with respect to ｜｜· ｜｜c or ｜｜L2· A numerical experiment shows that the method given in this paper is stable in the reproducing kernel space.

A NOVEL METHOD FOR NONLINEAR IMPULSIVE DIFFERENTIAL EQUATIONS IN BROKEN REPRODUCING KERNEL SPACE

In this article,a new algorithm is presented to solve the nonlinear impulsive differential equations.In the first time,this article combines the reproducing kernel method with the least squares method to solve the second-order nonlinear impulsive differential equations.Then,the uniform convergence of the numerical solution is proved,and the time consuming Schmidt orthogonalization process is avoided.The algorithm is employed successfully on some numerical examples.

The Expansion of the Function with Two Unknowns on the Reproducing Kernel Space

In this paper we make use of a special procedure on the repro ducing kernel space to give an expansion theorem for the function with two unkno wns and a surface approximation formula. The error of the surface possesses mono tonically decreasing and uniformly convergent characteristics in the sense of t he norm on the space.

ON APPROXIMATION BY REPRODUCING KERNEL SPACES IN WEIGHTED L^p SPACES

In this paper, we investigate the order of approximation by reproducing kernel spaces on （-1, 1） in weighted L^p spaces. We first restate the translation network from the view of reproducing kernel spaces and then construct a sequence of approximating operators with the help of Jacobi orthogonal polynomials, with which we establish a kind of Jackson inequality to describe the error estimate. Finally, The results are used to discuss an approximation problem arising from learning theory.

An Interpretable Denoising Layer for Neural Networks Based on Reproducing Kernel Hilbert Space and its Application in Machine Fault Diagnosis

Deep learning algorithms based on neural networks make remarkable achievements in machine fault diagnosis,while the noise mixed in measured signals harms the prediction accuracy of networks.Existing denoising methods in neural networks,such as using complex network architectures and introducing sparse techniques,always suffer from the difficulty of estimating hyperparameters and the lack of physical interpretability.To address this issue,this paper proposes a novel interpretable denoising layer based on reproducing kernel Hilbert space(RKHS)as the first layer for standard neural networks,with the aim to combine the advantages of both traditional signal processing technology with physical interpretation and network modeling strategy with parameter adaption.By investigating the influencing mechanism of parameters on the regularization procedure in RKHS,the key parameter that dynamically controls the signal smoothness with low computational cost is selected as the only trainable parameter of the proposed layer.Besides,the forward and backward propagation algorithms of the designed layer are formulated to ensure that the selected parameter can be automatically updated together with other parameters in the neural network.Moreover,exponential and piecewise functions are introduced in the weight updating process to keep the trainable weight within a reasonable range and avoid the ill-conditioned problem.Experiment studies verify the effectiveness and compatibility of the proposed layer design method in intelligent fault diagnosis of machinery in noisy environments.

Convergence analysis for complementary-label learning with kernel ridge regression

Complementary-label learning(CLL)aims at finding a classifier via samples with complementary labels.Such data is considered to contain less information than ordinary-label samples.The transition matrix between the true label and the complementary label,and some loss functions have been developed to handle this problem.In this paper,we show that CLL can be transformed into ordinary classification under some mild conditions,which indicates that the complementary labels can supply enough information in most cases.As an example,an extensive misclassification error analysis was performed for the Kernel Ridge Regression(KRR)method applied to multiple complementary-label learning(MCLL),which demonstrates its superior performance compared to existing approaches.

Minimax designs for linear regression models with bias in a reproducing kernel Hilbert space in a discrete set

Consider the design problem for estimation and extrapolation in approximately linear regression models with possible misspecification. The design space is a discrete set consisting of finitely many points, and the model bias comes from a reproducing kernel Hilbert space. Two different design criteria are proposed by applying the minimax approach for estimating the parameters of the regression response and extrapolating the regression response to points outside of the design space. A simulated annealing algorithm is applied to construct the minimax designs. These minimax designs are compared with the classical D-optimal designs and all-bias extrapolation designs. Numerical results indicate that the simulated annealing algorithm is feasible and the minimax designs are robust against bias caused by model misspecification.

ON APPROXIMATION BY SPHERICAL REPRODUCING KERNEL HILBERT SPACES

The spherical approximation between two nested reproducing kernels Hilbert spaces generated from different smooth kernels is investigated. It is shown that the functions of a space can be approximated by that of the subspace with better smoothness. Furthermore, the upper bound of approximation error is given.

Solving Neumann Boundary Problem with Kernel-Regularized Learning Approach

We provide a kernel-regularized method to give theory solutions for Neumann boundary value problem on the unit ball. We define the reproducing kernel Hilbert space with the spherical harmonics associated with an inner product defined on both the unit ball and the unit sphere, construct the kernel-regularized learning algorithm from the view of semi-supervised learning and bound the upper bounds for the learning rates. The theory analysis shows that the learning algorithm has better uniform convergence according to the number of samples. The research can be regarded as an application of kernel-regularized semi-supervised learning.

Iterative method of solving Cauchy problem with reproducing kernel for nonlinear hyperbolic equations

Presents the iterative method of solving Cauchy problem with reproducing kernel for nonlinear hyperbolic equations, and the application of the computational technique of reproducing kernel space to simplify, the iterative computation and increase the convergence rate and points out that this method is still effective. Even if the initial condition is discrete.

Attractive Multistep Reproducing Kernel Approach for Solving Stiffness Differential Systems of Ordinary Differential Equations and Some Error Analysis

In this paper,an efficient multi-step scheme is presented based on reproducing kernel Hilbert space(RKHS)theory for solving ordinary stiff differential systems.The solution methodology depends on reproducing kernel functions to obtain analytic solutions in a uniform formfor a rapidly convergent series in the posed Sobolev space.Using the Gram-Schmidt orthogonality process,complete orthogonal essential functions are obtained in a compact field to encompass Fourier series expansion with the help of kernel properties reproduction.Consequently,by applying the standard RKHS method to each subinterval,approximate solutions that converge uniformly to the exact solutions are obtained.For this purpose,several numerical examples are tested to show proposed algorithm’s superiority,simplicity,and efficiency.The gained results indicate that themulti-step RKHSmethod is suitable for solving linear and nonlinear stiffness systems over an extensive duration and giving highly accurate outcomes.

Reproducing wavelet kernel method in nonlinear system identification

By combining the wavelet decomposition with kernel method, a practical approach of universal multiscale wavelet kernels constructed in reproducing kernel Hilbert space （RKHS） is discussed, and an identification scheme using wavelet support vector machines （WSVM） estimator is proposed for nordinear dynamic systems. The good approximating properties of wavelet kernel function enhance the generalization ability of the proposed method, and the comparison of some numerical experimental results between the novel approach and some existing methods is encouraging.

New Implementation of Reproducing Kernel Method for Solving Functional-Differential Equations

In this paper, we apply the new algorithm of reproducing kernel method to give the approximate solution to some functional-differential equations. The numerical results demonstrate the accuracy of the proposed algorithm.

A Sparse Kernel Approximate Method for Fractional Boundary Value Problems

In this paper,the weak pre-orthogonal adaptive Fourier decomposition(W-POAFD)method is applied to solve fractional boundary value problems(FBVPs)in the reproducing kernel Hilbert spaces(RKHSs)W_(0)^(4)[0,1] and W^(1)[0,1].The process of the W-POAFD is as follows:(i)choose a dictionary and implement the pre-orthogonalization to all the dictionary elements;(ii)select points in[0,1]by the weak maximal selection principle to determine the corresponding orthonormalized dictionary elements iteratively;(iii)express the analytical solution as a linear combination of these determined dictionary elements.Convergence properties of numerical solutions are also discussed.The numerical experiments are carried out to illustrate the accuracy and efficiency of W-POAFD for solving FBVPs.

Differentially private SGD with random features

In the realm of large-scale machine learning,it is crucial to explore methods for reducing computational complexity and memory demands while maintaining generalization performance.Additionally,since the collected data may contain some sensitive information,it is also of great significance to study privacy-preserving machine learning algorithms.This paper focuses on the performance of the differentially private stochastic gradient descent(SGD)algorithm based on random features.To begin,the algorithm maps the original data into a lowdimensional space,thereby avoiding the traditional kernel method for large-scale data storage requirement.Subsequently,the algorithm iteratively optimizes parameters using the stochastic gradient descent approach.Lastly,the output perturbation mechanism is employed to introduce random noise,ensuring algorithmic privacy.We prove that the proposed algorithm satisfies the differential privacy while achieving fast convergence rates under some mild conditions.

THE SPARSE REPRESENTATION RELATED WITH FRACTIONAL HEAT EQUATIONS

This study introduces a pre-orthogonal adaptive Fourier decomposition(POAFD)to obtain approximations and numerical solutions to the fractional Laplacian initial value problem and the extension problem of Caffarelli and Silvestre(generalized Poisson equation).As a first step,the method expands the initial data function into a sparse series of the fundamental solutions with fast convergence,and,as a second step,makes use of the semigroup or the reproducing kernel property of each of the expanding entries.Experiments show the effectiveness and efficiency of the proposed series solutions.

On Reproducing Kernel Banach Spaces:Generic Definitions and Unified Framework of Constructions

Recently,there has been emerging interest in constructing reproducing kernel Banach spaces(RKBS)for applied and theoretical purposes such as machine learning,sampling reconstruction,sparse approximation and functional analysis.Existing constructions include the reflexive RKBS via a bilinear form,the semi-inner-product RKBS,the RKBS with?~1 norm,the p-norm RKBS via generalized Mercer kernels,etc.The definitions of RKBS and the associated reproducing kernel in those references are dependent on the construction.Moreover,relations among those constructions are unclear.We explore a generic definition of RKBS and the reproducing kernel for RKBS that is independent of construction.Furthermore,we propose a framework of constructing RKBSs that leads to new RKBSs based on Orlicz spaces and unifies existing constructions mentioned above via a continuous bilinear form and a pair of feature maps.Finally,we develop representer theorems for machine learning in RKBSs constructed in our framework,which also unifies representer theorems in existing RKBSs.

Conditional Kernel Covariance and Correlation

The conditional kernel correlation is proposed to measure the relationship between two random variables under covariates for multivariate data.Relying on the framework of reproducing kernel Hilbert spaces,we give the definitions of the conditional kernel covariance and conditional kernel correlation.We also provide their respective sample estimators and give the asymptotic properties,which help us construct a conditional independence test.According to the numerical results,the proposed test is more effective compared to the existing one under the considered scenarios.A real data is further analyzed to illustrate the efficacy of the proposed method.

Large Dynamic Covariance Matrix Estimation with an Application to Portfolio Allocation:A Semiparametric Reproducing Kernel Hilbert Space Approach

The estimation of high dimensional covariance matrices is an interesting and important research topic for many empirical time series problems such as asset allocation. To solve this dimension dilemma, a factor structure has often been taken into account. This paper proposes a dynamic factor structure whose factor loadings are generated in reproducing kernel Hilbert space(RKHS), to capture the dynamic feature of the covariance matrix. A simulation study is carried out to demonstrate its performance. Four different conditional variance models are considered for checking the robustness of our method and solving the conditional heteroscedasticity in the empirical study. By exploring the performance among eight introduced model candidates and the market baseline, the empirical study from 2001 to 2017 shows that portfolio allocation based on this dynamic factor structure can significantly reduce the variance, i.e., the risk, of portfolio and thus outperform the market baseline and the ones based on the traditional factor model.

A Gradient Iteration Method for Functional Linear Regression in Reproducing Kernel Hilbert Spaces

We consider a gradient iteration algorithm for prediction of functional linear regression under the framework of reproducing kernel Hilbert spaces.In the algorithm,we use an early stopping technique,instead of the classical Tikhonov regularization,to prevent the iteration from an overfitting function.Under mild conditions,we obtain upper bounds,essentially matching the known minimax lower bounds,for excess prediction risk.An almost sure convergence is also established for the proposed algorithm.