近红外光谱奇异样本剔除方法研究

Outlier Sample Eliminating Methods for Building Calibration Model of Near Infrared Spectroscopy Analysis

采用近红外光谱分析技术建立面粉校正模型,对面粉中灰分含量进行定量分析,并对异常样本进行剔除.试验中采用马氏距离法和蒙特卡洛采样法分别对异常样本进行了剔除,结果表明:用马氏距离法剔除异常样本,当权重系数为1.5,剔除样本数为3时,得到最好结果,相关系数(R2)为92.67,交互验证均方差RMSECV为0.048 5;MCCV法剔除异常样本,剔除样本数为3,得到最好结果,相关系数(R2)为94.64,交互验证均方差RMSECV为0.041 1.故马氏距离法剔除异常样本能在一定程度上提高校正模型的精度和预测精度,但MCCV法剔除异常样本后模型精度和预测精度优于马氏距离法.

The accuracy of the prediction model is affected by the near-infrared spectrum of flour and flour ash contents was quantitative analyzed. While the presence of outlier data seriously interfere with the reliability of the model, therefore, it is essential to identify and deal with the outlier samples to improve the predictive ability. Mahalanobis distance and the Monte Carlo cross validation （MCCV） methods were used to remove the outlier samples. When the weight coefficient was 1.5, excluding sample number was 3 with the former method it could get the best results, and the related coefficient （R2） was 92.67, crossvalidation mean square error （RMSECV） was 0. 048 5. While with the latter method the correlation coefficient （R2） was 94.64, cross-validation mean square error （RMSECV） was 0. 041 1. Therefore, Mahalanobis distance method can improve the calibration model and prediction accuracy to a certain extent, while the calibration model and prediction accuracy of MCCV without outliers samples was better than that of the Mahalanobis distance method.