变电站历史数据分布式存储系统设计与实现

Design and Implementation of a Distributed Storage System for Historical Substation Data

目前,变电站接入的数据类型越来越多,存在存储分散,结构和访问方式差异大,以及没有建立统一的存储和访问接口等问题。针对以上问题,提出一种基于Key-Value非关系型数据库的"Redis代理机+SSDB存储集群"的变电站历史数据分布式存储的实现方式,包括Redis代理服务器和SSDB存储集群,在代理服务器端通过历史数据统一导入工具,将不同厂家的变电站SCADA历史数据和非结构化历史数据统一存储在Key-Value网络化缓存库中,再通过代理服务程序,采用一致性哈希分片算法将其分布式转存至存储集群中,对于存储集群的每个存储子机通过LevelDB存储引擎将缓存数据存入磁盘文件中,实现对历史数据的分布式永久性存储。经实验室测试证明,开发出的变电站历史数据分布式存储系统的功能需求和性能指标完全达到预期目标。

At present, so many types of data are connected to substations that there appear such problems as dispersed storage, large difference in the respect of structure and access mode, and no existence of unified storage and access interface. In view of above problems, based on the Key-Value non-relational database, a method for realizing distributed storage of historical substation data, namely, Redis agent + SSDB storage cluster was presented in this paper. The system contained a Redis proxy server and a SSDB storage cluster. At the side of the proxy server, historical SCADA data and unstructured historical data of substations from different manufacturers were all stored in the Key-Value networked cache library with the help of a unified import tool for historical data. Then, through use of proxy service program, cached data was transferred to the storage cluster in the consistent Hash fragment algorithm. For each storage sub-machine of the storage cluster, the cached data was stored in the disk file through the LevelDB storage engine to realize distributed permanent storage of historical data. Laboratory tests proved that the functional requirement and performance indexes of the developed distributed data storage system for historical substation data reached the expected goal.