基于Hive的电力设备状态信息数据仓库

Data warehouse of electric power equipment condition information based on Hive

随着智能变电站的建设及其状态监测系统的发展,电力设备状态信息数据规模逐渐增大。针对现有电力数据仓库在海量状态数据存储查询和分析处理方面的不足,提出基于Hive的电力设备状态信息数据仓库及其多维数据快速查询与分析方法。通过对状态监测系统与生产管理系统(PMS)的分析,将电力设备静态信息与状态监测信息存储到Hive数据仓库中。设计了基于Hive的电力设备状态信息数据仓库的系统架构和海量状态数据存储结构,采用Hadoop分布式文件系统(HDFS)对数据进行分布式存储管理,Map Reduce作为海量数据查询分析的计算模式,HiveQL查询语言作为数据仓库的控制工具,并给出了数据仓库的工作过程。建立一个基于Hive的电力设备状态信息数据仓库实验平台,分别利用5个节点和10个节点的Hadoop集群进行测试,结果表明上述方法具有较好的扩展性,能满足大规模多维电力设备状态数据存储查询的需要。

With the continuous construction of smart substation and the development of its condition monitoring system, the data size of electric power equipment on condition monitoring is leaping. Aiming at deficiencies of current electric power data warehouse on massive condition monitoring data storage, query and analysis, a method of data warehouse based on Hive for fast query and analysis on multidimensional data is proposed. First, through analyzing condition monitoring system and production management system, the static information and condition monitoring information of electric power equipment are stored in Hive data warehouse. Second, the architecture of the data warehouse and storage structure of massive condition data are designed, adopting Hadoop distributed file system （HDFS） for distributed storage and management, MapReduce as computing model of massive data query and analysis, and Hive Query Language （HiveQL） as a control tool of data warehouse. The process of data warehouse is given respectively. Finally, an experimental platform of data warehouse for electric power equipment condition information based on Hive is established, results of multidimensional data queries on 5 nodes and 10 nodes cluster show that this method has good scalability, and can meet the needs of fast query on large scaled multidimensional condition data of electric power equipment.