多维数据立方体的分块与压缩设计

Design on Block and Compression of Multidimensional Data Cube

目前提出的关于多维数组存储组织的有效方法,没有有效解决存储空间的浪费和存储维内部层次信息问题,导致存储浪费。采用Fragment分块方法将高维空间进行降维存储,分别分为稀疏维和密集维,数据块建立在稀疏维成员组合的基础之上,即将稀疏维相同的度量数据存储在一个数据块中,每个数据块有唯一的标识。对多维数据立方体进行了分块处理,并获得了每个数据块的标识。对于是否需要创建该数据块,只需要在生成数据文件时判断该数据块是否为空,若为空则不需要创建该数据块;若不为空,则创建该数据块。最后给出多维数据立方体的压缩算法。

The methods proposed by the multidimensional array storage organization have no effective solution to solve the storage space waste and internal hierarchical information storage. This paper adopts Fragment partition method to fragment the block to high -dimensional space dimension reduction of storage, which is respectively divided into sparse and dense, block of data is set up based on sparse group, i.e. the same sparse dimension measurement data is stored in a data block, each block has a unique ID. The block processing of multidimensional data cube is conducted and the identity, of each data block is obtained. For the data block creating, the situation, whether the data block is empty or not, should be judged when the data file is generated. The data block is not be required if it is empty, and if not null, then the data block should be created. The multidimensional data cube compression algorithm is given as well.