1种基于自助法的奇异值检测方法

A bootstrap-based outlier detection method

如何检测数据集中的奇异值仍然是多元校正中的1个重要的问题。对于化学计量学研究者来说,找到1个普遍适用的方法仍然是1个重要的任务。本文的目的是介绍1种较新的基于自助法的奇异值检测方法。本法以内部学生化残差为基准,用自助法对相关变量进行估计,并采用刀切-自助法对估计值进行评价。它不要求回归模型的残差服从正态分布,因而适用于大部分回归分析中的奇异值检测。本文中采用烟草和玉米样本的近红外光谱数据对该法进行验证,结果表明,采用基于自助法的奇异值检测方法剔除奇异样品后,模型的预测误差减小15%,优于学生化残差-杠杆值法和稳健偏最小二乘法。我们还在玉米近红外光谱的基础上,进行了奇异样品数的模拟研究,并采用该法进行检验。结果表明,当奇异样品的数量少于总样品数的10%时,该方法的表现较其它2种方法好。所以,基于自助法的奇异值检测方法是1种有效的方法。

How to detect outliers in a data set is still an important problem in multivariate calibration.To chemometricians,finding a widely applicable method is still a task.The aim of this paper is to introduce a bootstrap-based outlier detection.The inner Studentized residuals was taken as an index.A bootstrap was applied to estimate model parameters.And the jackknife-after-bootstrap was applied to evaluate the parameters.A tobacco data set and a corn data set were used to demonstrate the performance of the method.The results showed that a 15%improvement of the performance of a PLS model was obtained after the outliers were detected and removed by the method.This result is better than the ordinary outlier detection method based on the Studentized residuals and robust SIMPLS.On the other hand,we made simulation based on the corn data.A number of outliers were introduced into the data and the method was applied to detect the outliers.The results show that the performance of the method is relatively better than two other methods when the percentage of outliers is less than 10%.So,the bootstrap-based outlier detection is a valid method.