一种高压缩率的空间数据序列化方法在输电线路通道数据分析中应用

A High Compression Rate Spatial Data Serialization Method

近年来,随着电网信息化的发展,电网空间数据的数据量飞速增长。海量的电网空间数据逐渐为硬盘、内存、网络带宽等硬件资源带来了占用压力。传统的空间数据多按照开源组织OGC提出的WKB规范压缩为字节码进行存储、传输和计算,而WKB的定长压缩方式带来了较大的空间冗余,其压缩率不足以满足日益增长的数据要求。另一方面,空间数据往往还关联有有编号、名称等属性,WKB规范不包含属性的序列化方案,需要将属性与空间数据分别传输,带来了数据的二次查询和关联上的麻烦。本文基于压缩率更高的protobuf规范,设计并实现了一种高压缩率的空间数据序列化方法。该方法结合不定长压缩,冗余属性标记等方案,极大地压缩了数据大小,相比WKB规范,节省大约20%的空间。同时,本序列化方法也支持附加属性,免去了属性和空间数据的关联过程。

In recent years,with the development of power grid informatization,the data volume of power grid spatial data is increasing rapidly.Massive spatial data of power grid gradually brings occupation pressure to hardware resources such as hard disk,memory and network bandwidth.Traditional spatial data are mostly compressed for bytecode storage,transmission and computation according to the WKB specification proposed by the open source organization OGC,while WKB’s fixed-length compression method brings large spatial redundancy,and its compression rate is not enough to meet the increasing data requirements.On the other hand,spatial data is often associated with attributes such as numbers and names.WKB specification does not include the serialization scheme of attributes,so it is necessary to transmit attributes and spatial data separately,which brings trouble in the secondary query and association of data.Based on the protobuf specification with higher compression rate,this paper designs and implements a spatial data serialization method with high compression rate.This method combines indefinite length compression,redundant attribute marking and other schemes,greatly reduces the data size,and saves about 20%space compared with WKB specification.At the same time,this serialization method also supports additional attributes,eliminating the association process between attributes and spatial data.