An Interpretable Denoising Layer for Neural Networks Based on Reproducing Kernel Hilbert Space and its Application in Machine Fault Diagnosis

Deep learning algorithms based on neural networks make remarkable achievements in machine fault diagnosis,while the noise mixed in measured signals harms the prediction accuracy of networks.Existing denoising methods in neural networks,such as using complex network architectures and introducing sparse techniques,always suffer from the difficulty of estimating hyperparameters and the lack of physical interpretability.To address this issue,this paper proposes a novel interpretable denoising layer based on reproducing kernel Hilbert space(RKHS)as the first layer for standard neural networks,with the aim to combine the advantages of both traditional signal processing technology with physical interpretation and network modeling strategy with parameter adaption.By investigating the influencing mechanism of parameters on the regularization procedure in RKHS,the key parameter that dynamically controls the signal smoothness with low computational cost is selected as the only trainable parameter of the proposed layer.Besides,the forward and backward propagation algorithms of the designed layer are formulated to ensure that the selected parameter can be automatically updated together with other parameters in the neural network.Moreover,exponential and piecewise functions are introduced in the weight updating process to keep the trainable weight within a reasonable range and avoid the ill-conditioned problem.Experiment studies verify the effectiveness and compatibility of the proposed layer design method in intelligent fault diagnosis of machinery in noisy environments.

Minimax designs for linear regression models with bias in a reproducing kernel Hilbert space in a discrete set

Consider the design problem for estimation and extrapolation in approximately linear regression models with possible misspecification. The design space is a discrete set consisting of finitely many points, and the model bias comes from a reproducing kernel Hilbert space. Two different design criteria are proposed by applying the minimax approach for estimating the parameters of the regression response and extrapolating the regression response to points outside of the design space. A simulated annealing algorithm is applied to construct the minimax designs. These minimax designs are compared with the classical D-optimal designs and all-bias extrapolation designs. Numerical results indicate that the simulated annealing algorithm is feasible and the minimax designs are robust against bias caused by model misspecification.

ON APPROXIMATION BY SPHERICAL REPRODUCING KERNEL HILBERT SPACES

The spherical approximation between two nested reproducing kernels Hilbert spaces generated from different smooth kernels is investigated. It is shown that the functions of a space can be approximated by that of the subspace with better smoothness. Furthermore, the upper bound of approximation error is given.

Solution of Klein-Gordon Equation by a Method of Lines Using Reproducing Kernel Hilbert Space Method

This paper presents a method of lines solution based on the reproducing kernel Hilbert space method to the nonlinear one-dimensional Klein-Gordon equation that arises in many scientific fields areas.Our method uses discretization of the partial derivatives of the space variable to get a system of ODEs in the time variable and then solve the system of ODEs using reproducing kernel Hilbert space method.Consider two examples to validate the proposed method.Compare the results with the exact solution by calculating the error norms L_(2) and L_(∞) at various time levels.The results show that the presented scheme is a systematic,effective and powerful technique for the solution of Klein-Gordon equation.

Large Dynamic Covariance Matrix Estimation with an Application to Portfolio Allocation:A Semiparametric Reproducing Kernel Hilbert Space Approach

The estimation of high dimensional covariance matrices is an interesting and important research topic for many empirical time series problems such as asset allocation. To solve this dimension dilemma, a factor structure has often been taken into account. This paper proposes a dynamic factor structure whose factor loadings are generated in reproducing kernel Hilbert space(RKHS), to capture the dynamic feature of the covariance matrix. A simulation study is carried out to demonstrate its performance. Four different conditional variance models are considered for checking the robustness of our method and solving the conditional heteroscedasticity in the empirical study. By exploring the performance among eight introduced model candidates and the market baseline, the empirical study from 2001 to 2017 shows that portfolio allocation based on this dynamic factor structure can significantly reduce the variance, i.e., the risk, of portfolio and thus outperform the market baseline and the ones based on the traditional factor model.

A Gradient Iteration Method for Functional Linear Regression in Reproducing Kernel Hilbert Spaces

We consider a gradient iteration algorithm for prediction of functional linear regression under the framework of reproducing kernel Hilbert spaces.In the algorithm,we use an early stopping technique,instead of the classical Tikhonov regularization,to prevent the iteration from an overfitting function.Under mild conditions,we obtain upper bounds,essentially matching the known minimax lower bounds,for excess prediction risk.An almost sure convergence is also established for the proposed algorithm.

Convergence analysis for complementary-label learning with kernel ridge regression

Complementary-label learning(CLL)aims at finding a classifier via samples with complementary labels.Such data is considered to contain less information than ordinary-label samples.The transition matrix between the true label and the complementary label,and some loss functions have been developed to handle this problem.In this paper,we show that CLL can be transformed into ordinary classification under some mild conditions,which indicates that the complementary labels can supply enough information in most cases.As an example,an extensive misclassification error analysis was performed for the Kernel Ridge Regression(KRR)method applied to multiple complementary-label learning(MCLL),which demonstrates its superior performance compared to existing approaches.

Solving Neumann Boundary Problem with Kernel-Regularized Learning Approach

We provide a kernel-regularized method to give theory solutions for Neumann boundary value problem on the unit ball. We define the reproducing kernel Hilbert space with the spherical harmonics associated with an inner product defined on both the unit ball and the unit sphere, construct the kernel-regularized learning algorithm from the view of semi-supervised learning and bound the upper bounds for the learning rates. The theory analysis shows that the learning algorithm has better uniform convergence according to the number of samples. The research can be regarded as an application of kernel-regularized semi-supervised learning.

指数权的加权Lebesgue空间中拟齐次核最优Hilbert型积分不等式的参数条件及应用

首先,引入拟齐次核的概念,讨论权函数为指数函数的加权Lebesgue空间中具有拟齐次核的Hilbert型积分不等式;其次,利用权系数方法及若干分析技巧,给出最优Hilbert型积分不等式的等价参数条件,并获得了最佳常数因子的计算公式;最后,讨论其在算子理论中的应用.

加权Hilbert型空间中超齐次核离散算子的最佳搭配参数及范数计算

引入超齐次核概念,利用权系数方法,讨论具有超齐次核离散算子在加权Hilbert型空间中的有界性及算子范数,得到该类算子最佳搭配参数的充分必要条件和算子范数的计算公式,统一了齐次核,广义齐次核及若干非齐次核情形的相关结果.

基于再生核Hilbert空间小波核函数支持向量机的高光谱遥感影像分类

针对支持向量机用于高光谱遥感影像分类存在的分类精度不高、参数选择困难等问题,提出一种再生核Hilbert空间的小波核。其可以逼近任意非线性函数,能够有效改进参数估计的效果,进而实现基于再生核Hilbert空间的小波核函数支持向量机(小波支持向量机)。并选取北京昌平地区的国产高光谱数据operational modular imaging spec-trometer II(OMIS II)和意大利Pavia大学ROSIS高光谱数据进行试验。结果表明,应用Coiflet小波核函数时能获得较高分类精度。

Attractive Multistep Reproducing Kernel Approach for Solving Stiffness Differential Systems of Ordinary Differential Equations and Some Error Analysis

In this paper,an efficient multi-step scheme is presented based on reproducing kernel Hilbert space(RKHS)theory for solving ordinary stiff differential systems.The solution methodology depends on reproducing kernel functions to obtain analytic solutions in a uniform formfor a rapidly convergent series in the posed Sobolev space.Using the Gram-Schmidt orthogonality process,complete orthogonal essential functions are obtained in a compact field to encompass Fourier series expansion with the help of kernel properties reproduction.Consequently,by applying the standard RKHS method to each subinterval,approximate solutions that converge uniformly to the exact solutions are obtained.For this purpose,several numerical examples are tested to show proposed algorithm’s superiority,simplicity,and efficiency.The gained results indicate that themulti-step RKHSmethod is suitable for solving linear and nonlinear stiffness systems over an extensive duration and giving highly accurate outcomes.

基于再生核Hilbert空间的非线性信道均衡算法

在高速无线通信领域,为消除码间干扰(ISI)必须研究非线性信道均衡技术。基于再生核希尔伯特空间(RKHS)研究非线性信道的自适应均衡算法。首先基于非线性维纳模型提出均衡器的结构,基于RKHS引入核方法,与仿射投影算法(APA)相结合推导出核仿射投影算法(KAPA),再通过引入松弛因子得到改进的KAPA算法。用蒙特卡罗法对提出的自适应算法进行仿真,从收敛性能、误码率(BER)、跟踪能力、计算复杂度等方面与其他算法做比较。在不增加计算复杂度的情况下,极大降低了误码率,非常适合时变非线性信道均衡的应用。

再生核Hilbert空间中的一种插值方法

在给定的再生核Hilbert空间中,利用再生核的性质,通过再生核函数的线性组合得到插值基函数,从而构造了插值函数,并给出误差估计和数值算例.该方法是这个再生核Hilbert空间中一种新的插值方法,计算量小,收敛速度较快,便于实际应用.

具有再生核Hilbert空间中紧的偏微分算子

用再生核函数来刻画再生核空间中算子的性质,是研究再生核空间性质的一个重要方法.在本文中,研究了具有再生核的多元整函数Hilbert空间的基本性质,着重讨论了偏微分算子在该空间上的紧性,给出了一个用再生核函数刻画的偏微分算子是紧算子的充分必要条件,从而在具有再生核的多元整函数Hilbert空间上推广了已有的结果.

以{e_i}_i^n=1为正交基的再生核Hilbert空间

再生核Hilbert空间首先是一个Hilbert空间,再生核方法(RKHS method)为研究Hilbert空间提供了一个有力的数学工具,核函数具有许多优良的性质,可以通过这些性质来刻画整个Hilbert空间。笔者主要研究了以{ei}in=1为正交基的再生核Hilbert空间H中的核函数的一些性质,并通过这些性质简要的描述了Hilbert空间H与它的核函数之间的关系。

Reproducing wavelet kernel method in nonlinear system identification

By combining the wavelet decomposition with kernel method, a practical approach of universal multiscale wavelet kernels constructed in reproducing kernel Hilbert space （RKHS） is discussed, and an identification scheme using wavelet support vector machines （WSVM） estimator is proposed for nordinear dynamic systems. The good approximating properties of wavelet kernel function enhance the generalization ability of the proposed method, and the comparison of some numerical experimental results between the novel approach and some existing methods is encouraging.

A Sparse Kernel Approximate Method for Fractional Boundary Value Problems

In this paper,the weak pre-orthogonal adaptive Fourier decomposition(W-POAFD)method is applied to solve fractional boundary value problems(FBVPs)in the reproducing kernel Hilbert spaces(RKHSs)W_(0)^(4)[0,1] and W^(1)[0,1].The process of the W-POAFD is as follows:(i)choose a dictionary and implement the pre-orthogonalization to all the dictionary elements;(ii)select points in[0,1]by the weak maximal selection principle to determine the corresponding orthonormalized dictionary elements iteratively;(iii)express the analytical solution as a linear combination of these determined dictionary elements.Convergence properties of numerical solutions are also discussed.The numerical experiments are carried out to illustrate the accuracy and efficiency of W-POAFD for solving FBVPs.

几类与布朗运动有关的高斯过程的再生核Hilbert空间

研究了带线性漂移的布朗运动、Demeaned布朗运动以及Demeaned布朗桥的再生核Hilbert空间。首先,对于带线性漂移的布朗运动,利用其协方差函数进行了Karhunen-Loève(KL)展开,并用特征函数作为基函数构造出了再生核Hilbert空间。其次,对于Demeaned布朗运动,给出了其协方差函数,定义了相应的内积,构造出了再生核Hilbert空间。最后,对于Demeaned布朗桥,给出了其协方差函数,定义了相应的内积,构造出了再生核Hilbert空间。

New Implementation of Reproducing Kernel Method for Solving Functional-Differential Equations

In this paper, we apply the new algorithm of reproducing kernel method to give the approximate solution to some functional-differential equations. The numerical results demonstrate the accuracy of the proposed algorithm.