生产数据校验加载方法研究与实践

CHECKING AND LOADING METHODS OF PRODUCTION DATA

对于海量生产数据的校验与加载,由于数据量庞大和获取数据的途径不同,其内容、格式、质量参差不齐,经常会遇到数据格式不能转换或格式转换后信息丢失等棘手问题,并且数据质量难以控制,人工校验费时、费力且准确率低。故以西南油气田A2系统历史数据加载为实例,研讨数据校验加载流程,阐述如何开发数据质控平台,并利用该平台和数据内在的逻辑规则,对生产数据进行高效准确的校验和加载。

In checking and loading massive production data,because the huge data and their various acquiring channels result in irregularities in content,format and quality,some confused problems have been encountered,including unable conversion of data format or information loss after converting,data quality hard to being controlled,and time- consuming manual checking with a lower accuracy. According to the historical data loading of A2 system of PetroChina Southwest Oil and Gasfield Company,this paper discusses the data checking and loading processes,and illustrates how to develop a platform to control data quality. The efficient and accurate checking and loading of production data can be obtained by means of adopting logic rules between platform and data.