Data organization requires high efficiency for large amount of data applied in the digital mine system. A new method of storing massive data of block model is proposed to meet the characteristics of the database, incl...Data organization requires high efficiency for large amount of data applied in the digital mine system. A new method of storing massive data of block model is proposed to meet the characteristics of the database, including ACID-compliant, concurrency support, data sharing, and efficient access. Each block model is organized by linear octree, stored in LMDB（lightning memory-mapped database）. Geological attribute can be queried at any point of 3D space by comparison algorithm of location code and conversion algorithm from address code of geometry space to location code of storage. The performance and robustness of querying geological attribute at 3D spatial region are enhanced greatly by the transformation from 3D to 2D and the method of 2D grid scanning to screen the inner and outer points. Experimental results showed that this method can access the massive data of block model, meeting the database characteristics. The method with LMDB is at least 3 times faster than that with etree, especially when it is used to read. In addition, the larger the amount of data is processed, the more efficient the method would be.展开更多
With volume size increasing, it is necessary to develop a highly efficient compression algorithm, which is suitable for progressive refinement between the data server and the browsing client. For three-dimensional lar...With volume size increasing, it is necessary to develop a highly efficient compression algorithm, which is suitable for progressive refinement between the data server and the browsing client. For three-dimensional large volume data, an efficient hierarchical algorithm based on wavelet compression was presented, using intra-band dependencies of wavelet coefficients. Firstly, after applying blockwise hierarchical wavelet decomposition to large volume data, the block significance map was obtained by using one bit to indicate significance or insignificance of the block. Secondly, the coefficient block was further subdivided into eight sub-blocks if any significant coefficient existed in it, and the process was repeated, resulting in an incomplete octree. One bit was used to indicate significance or insignificance, and only significant coefficients were stored in the data stream. Finally, the significant coefficients were quantified and compressed by arithmetic coding. The experimental results show that the proposed algorithm achieves good compression ratios and is suited for random access of data blocks. The results also show that the proposed algorithm can be applied to progressive transmission of 3D volume data.展开更多
基金Projects(41572317,51374242)supported by the National Natural Science Foundation of ChinaProject(2015CX005)supported by the Innovation Driven Plan of Central South University,China
文摘Data organization requires high efficiency for large amount of data applied in the digital mine system. A new method of storing massive data of block model is proposed to meet the characteristics of the database, including ACID-compliant, concurrency support, data sharing, and efficient access. Each block model is organized by linear octree, stored in LMDB（lightning memory-mapped database）. Geological attribute can be queried at any point of 3D space by comparison algorithm of location code and conversion algorithm from address code of geometry space to location code of storage. The performance and robustness of querying geological attribute at 3D spatial region are enhanced greatly by the transformation from 3D to 2D and the method of 2D grid scanning to screen the inner and outer points. Experimental results showed that this method can access the massive data of block model, meeting the database characteristics. The method with LMDB is at least 3 times faster than that with etree, especially when it is used to read. In addition, the larger the amount of data is processed, the more efficient the method would be.
基金Supported by Natural Science Foundation of China (No. 60373061).
文摘With volume size increasing, it is necessary to develop a highly efficient compression algorithm, which is suitable for progressive refinement between the data server and the browsing client. For three-dimensional large volume data, an efficient hierarchical algorithm based on wavelet compression was presented, using intra-band dependencies of wavelet coefficients. Firstly, after applying blockwise hierarchical wavelet decomposition to large volume data, the block significance map was obtained by using one bit to indicate significance or insignificance of the block. Secondly, the coefficient block was further subdivided into eight sub-blocks if any significant coefficient existed in it, and the process was repeated, resulting in an incomplete octree. One bit was used to indicate significance or insignificance, and only significant coefficients were stored in the data stream. Finally, the significant coefficients were quantified and compressed by arithmetic coding. The experimental results show that the proposed algorithm achieves good compression ratios and is suited for random access of data blocks. The results also show that the proposed algorithm can be applied to progressive transmission of 3D volume data.
