模型诊断用于近红外光谱建模校正集中奇异样本的识别

Outlier Detection for Multivariate Calibration in Near Infrared Spectroscopic Analysis by Model Diagnostics

由于校正集样本的质量决定校正模型的质量,校正集中奇异样本的检测在多元校正建模中具有非常重要的意义。本研究建立了一种用于近红外光谱多元校正建模时校正集中奇异样本的检测方法。本方法基于奇异样本的定义和偏最小二乘方法的原理,通过考察每个校正集样本在模型的每个因子(或主成分)中对模型的贡献,将与多数样本表现不同的样本识别为奇异样本。采用218个橘汁样本构成的近红外光谱数据进行了分析,结果表明,校正集中存在6个奇异样本,扣除奇异样本后,校正集的交叉验证均方根误差由16.870减小为4.809,预测集的均方根误差从3.688减小为3.332。

Outlier detection is an important task in multivariate calibration because the quality of a calibration model is determined by that of the calibration data. An outlier detection method is proposed for near infrared （NIR） spectral analysis. The method is based on the definition of outlier and the principle of partial least squares （PLS） regression, i. e. , an outlier in a dataset behaves differently from the rest, and the prediction result of a PLS model is an accumulation of several independent latent variables. Therefore, the proposed method builds a PLS model with a calibration dataset, and then the contribution of each latent variable is investigated. Outliers can be detected by comparing these contributions. An NIR spectral dataset of orange juice samples is adopted for testing the method. Six outliers are detected in the calibration set. The root mean squared error of cross validation （RMSECV） becomes to 4. 809 from 16.870 and the root mean squared error of prediction （RMSEP） becomes to 3. 332 from 3. 688 after the removal of the outliers. Compared with a robust regression method, the result of the proposed method seems more reasonable.