基于机器学习方法的探空数据质量控制研究

Research on Quality Control of Sounding Data Based on Machine Learning Method

【目的】建立一套基于机器学习方法的探空数据的自动化质量控制检测算法模型,为提高大气探空数据质量控制的效率及质量提供参考。【方法】本模型使用大气探空观测原始数据,以三西格玛准则完成初步质控,基于三西格玛准则检测结果标记生成机器学习数据集,应用XGBoost机器学习算法完成最终质控模型的构建。【结果】实现了对大气探空观测数据温度、气压、湿度、仰角、方位角、斜距素数的异常数据检测,模型异常数据检测精确率96.7%,识别率比人工检测提高了43.5%。【结论】模型对要素异常值检测具有较好的效果,较人工识别性能有明显提升。

[Purposes]In order to improve the efficiency and quality of atmospheric sounding data quality control,this study established an automated quality control detection algorithm model for sounding data based on machine learning methods.[Methods]This model uses atmospheric sounding observation raw data,completes preliminary quality control using the Three Sigma criterion,generates machine learning datasets based on the Three Sigma criterion detection results,and applies XGBoost machine learning al-gorithm to complete the construction of the final quality control model.[Findings]The model achieved the detection of abnormal data in atmospheric sounding observation data such as temperature,pressure,humidity,elevation,azimuth,and diagonal prime.The test results showed that the accuracy of abnormal data detection in the model was 96.7%,and the recognition rate was improved by 43.5%compared to manual detection.[Conclusions]The model has a good effect on the detection of element outlier,which is significantly improved compared with the performance of manual identification.