非参数估计方法

Survey of non-parametric estimation methods

为了解决函数估计问题,首先讨论了传统的参数回归方法.由于传统方法需要先验知识来决定参数模型,因此不稳健,且对模型敏感.因此,引入了基于数据驱动的非参数方法,无需任何先验知识即可对未知函数进行估计.本文主要介绍最新的8种非参数回归方法:核方法、局部多项式回归、正则化方法、正态均值模型、小波方法、超完备字典、前向神经网络、径向基函数网络.比较了不同的算法,给出算法之间的相关性与继承性.最后,将算法推广到高维情况,指出面临计算的维数诅咒与样本的维数诅咒两个问题.通过研究指出前者可以通过智能优化算法求解,而后者是问题固有的.

In order to solve the problem of function estimation,we first discuss traditional parametric regression method.Since it needs a priori knowledge to determine the model,the parametric method is not robust and is model-sensitive.Thus,data-driven non-parametric method is introduced,which needs not any a prior knowledge to estimate the unknown function.Eight major non-parametric methods are discussed as kernel method,local polynomial regression,regularization method,normal mean model,wavelet method,overcomplete dictionary,forward neural network,and radial basis function network.These algorithms are compared,and their coherence and inheritance are investigated.Finally,generalize the algorithms to high dimensionality and point out two problems as curse of dimensionality of computation and sample.The former can be settled down by intelligent methods while the latter is problem intrinsic.