能源行业结构化数据质量评估研究

Research on Quality Assessment of Structured Data in Energy Industry

【目的】随着能源行业海量数据逐步接入区域能源数据中心,能源数据质量评估与修复工作日益重要。为解决海量能源行业数据中存在的结构化数据质量低、异常数据修复难度大的问题,开展能源行业结构化数据质量评估,对提高能源数据管理水平具有积极意义。【方法】通过Strong-Wang框架搭建能源行业结构化数据质量检查框架,实现能源行业结构化数据的质量规则类型和数据质量特性的标准化定义;通过构建元数据质量评估规则,实现能源行业结构化数据的多维度评估;通过戴明环方法“计划-执行-检查-处理”的全生命周期数据质量管理,实现能源行业结构化数据质量管理与异常数据修复。【结果】在某城市能源数据中心进行实际应用,电、水、气、热等类型的能源数据质量平均修复准确率达98.57%。【结论】该方法有效分析了能源行业结构化数据质量问题,实现了结构化数据质量的高效修复。

[Purposes]With the gradual integration of massive data from the energy industry into regional energy data centers,energy data quality assessment and restoration work is becoming increasingly important.In order to solve the problem of low quality of structured data and difficulty in repairing abnormal data in massive energy industry data,it is of positive significance to carry out quality assessment of structured data in the energy industry and improve the level of energy data management.[Methods]The structured data quality inspection framework of the energy industry was built through the Strong-Wang framework to realize the standardized definition of the quality rule types and data quality characteristics of structured data of the energy industry;The multi-dimensional evaluation of structured data in the energy industry is realized by constructing metadata quality evaluation rules;Through the full life cycle data quality management of Deming Cycle"plan-do-check-act",structured data quality management and abnormal data repair in the energy industry are realized.[Findings]In practical application in an energy data center in a certain city,the average repair accuracy of energy data quality for types such as electricity,water,gas,and heat reached 98.57%.[Conclusions]This method effectively analyzes the quality problems of structured data in the energy industry,and realizes the efficient repair of structured data quality.