基于HDFS的电力设备监测数据存储设计研究

Research on data storage design of power equipment monitoring based on HDFS

随着电网规模的日趋复杂以及电网规模的快速增长,电力生产与信息化有了深度的融合,电网监测数据变得日益庞大,电力设备在线监控系统的数据存储面临着巨大的挑战。针对电力设备监测数据存储存在的问题,采用HDFS对电力设备监测数据存储设计进行研究,研究了基于HDFS的电力设备监测数据存储系统构架,系统构架主要由计算层、数据接入和存储层组成;提出了一致性哈希存储算法,该算法优化了HDFS的数据分布策略;分析了HDFS数据块的尺寸优化,得出了负载均衡和数据块的关系;进行了HDFS性能基准测试,分析出监测数据规模与文件大小对集群I/O性能的影响,研究为实现高效、统一、安全的电力设备监控数据的存储提供一定借鉴意义。

With the rapid development of power grid scale and the rapid growth of power grid scale,power production and information technology have deep integration,power grid monitoring data becomes increasingly large,power equipment online monitoring system data storage is facing great challenges.In view of the problems existing in the storage of the monitoring data of the electric power equipment,HDFS was used to study the data storage system of power equipment monitoring data,and the architecture of data storage system based on HDFS was studied.The system architecture was mainly composed of computing layer,data access and storage layer.A consistent hash algorithm was proposed.The HDFS data distribution strategy was analyzed,the size of HDFS data block was analyzed,and the relationship between load balancing and data block was obtained.The HDFS performance benchmark test was carried out to analyze the influence of monitoring data size and file size on cluster I/O performance,research for the realization of efficient,unified,safe monitoring of power equipment to provide some reference data storage.