电力调度大数据应用平台系统技术研究

Research on the technology of big data application platform system for power dispatching

为解决智能电网的发展中电网运行和设备检测或监测数据、电力企业管理数据、电力企业营销等数据海量的增加带来的不同业务系统之间分散地开发、运行和管理,系统数据存储结构独立,带来数据多源、格式不一致,数据准确性、实时性不强,数据质量不高,缺乏统一的数据规范等问题,本文利用Hadoop的分布式文件系统HDFS和并行处理框架MapReduce的工作原理,搭建电网调度大数据应用平台系统,解决了不同业务系统之间的数据不能及时共享、访问、管理与分析挖掘等问题。采用数据清洗数据,解决数据质量不高的问题。搭建电网调度大数据应用平台系统,既能实现跨专业、跨部门的多维度关联分析,又能满足海量的智能电网数据存储和数据处理需求,并具有强大的伸缩性,可扩展为电网实现安全、可靠、经济、高效地运行提供保障。

In order to solve the problems in the development of smart grid,such as power grid operation and equipment detection or monitoring data,power enterprise management data,power enterprise marketing,etc,the increase of massive data brings about the development,operation and management of different business systems in a decentralized manner.The data storage structure of the system is independent,which leads to the problems of multi-source data,inconsistent format,low accuracy and real-time performance of data,low quality of data and lack of unified data specification.This article uses the working principle of Hadoop's distributed file system HDFS and parallel processing framework MapReduce to build the grid dispatching big data application platform system,which solves the problems of data sharing,access,management and analysis mining between different business systems in time.Data cleaning is used to solve the problem of low data quality.The construction of power grid dispatching big data application platform system can not only realize multi-dimensional correlation analysis of cross discipline and cross department,but also meet the requirements of massive smart grid data storage and data processing.It has strong scalability and can be extended to provide guarantee for the safe,reliable,economic and efficient operation of power grid.