基于分布式文件系统电力大数据存储实现

Realization of power big data storage based on distributed file system

基于Hadoop存储架构设计了电网非结构化数据管理平台,主要包括存储分析与搜索读取2大模块,整合HDFS、Hbase等存储设备,利用HDFS实现海量数据的快速读写,采用基于ZooKeeper及Solr搭建的开源分布式搜索引擎SolrCloud实现数据检索,提供了高效便捷的智能化管理手段。

This paper designs an unstructured data management platform for power grids based on the Hadoop storage architecture.It mainly includes two modules:storage analysis and search and reading.It integrates storage devices such as HDFS and Hbase.It uses HDFS to achieve rapid reading and writing of massive data and adopts an open-source distributed search engine SolrCloud built by ZooKeeper and Solr to implement data retrieval,providing an effective and convenient method for smart management.