基于神经正切核的小数据集回归任务

Neural Tangent Kernel based regression task for small data sets

回归是常见的一类任务,两类特殊的回归模型:支持向量回归(SVR)与核岭回归(KRR)通过核函数解决数据在原始空间线性不可分的问题。一种新型核函数(NTK)被提出用于拟合无限宽神经网络的训练过程,相关研究显示NTK利于处理小数据集。选取多领域数据集在两种模型中比较NTK与常用核的性能,并对NTK进行了鲁棒性研究。结果表明NTK-SVR模型在部分数据集上取得了2.5%~20%的提升。

Regression is a common class of tasks,and two special types of regression models,which means Support Vector Regression(SVR)and Kernel Ridge Regression(KRR)solve the linearly inseparable problem of data in the original space by means of kernel functions.A new Neural Tangent Kernel(NTK)has been proposed to fit the training process of infinitely wide neural networks,and the related studies have shown that NTK is beneficial for handling small datasets.A multi-domain data set is selected to compare the performance of NTK with commonly used kernels in two models,and the robustness of NTK is investigated.The results show that the NTK-SVR model achieves a 2.5%~20%improvement on some data sets.