利用表格分析的方式,对比激光雷达系统(LIDAR)标准文件格式LAS version 2.0与旧版本verison1.1之间的异同,分析2.0版本的新特性,便于对LAS格式的理解和应用。新的定义标准更加适应LIDAR硬件间的通用性,给予用户更多的扩展空间,为LIDAR...利用表格分析的方式,对比激光雷达系统(LIDAR)标准文件格式LAS version 2.0与旧版本verison1.1之间的异同,分析2.0版本的新特性,便于对LAS格式的理解和应用。新的定义标准更加适应LIDAR硬件间的通用性,给予用户更多的扩展空间,为LIDAR系统的应用提供高质量的文件交换基础。展开更多
We describe an efficient and easily applicable data deduplication framework with heuristic prediction based adaptive block skipping for the real-world dataset such as disk images to save deduplication related overhead...We describe an efficient and easily applicable data deduplication framework with heuristic prediction based adaptive block skipping for the real-world dataset such as disk images to save deduplication related overheads and improve deduplication throughput with good deduplication efficiency maintained. Under the framework, deduplication operations are skipped for data chunks determined as likely non-duplicates via heuristic prediction, in conjunction with a hit and matching extension process for duplication identification within skipped blocks and a hysteresis mechanism based hash indexing process to update the hash indices for the re-encountered skipped chunks. For performance evaluation, the proposed framework was integrated and implemented in the existing data domain and sparse indexing deduplication algorithms. The experimental results based on a real-world dataset of 1.0 TB disk images showed that the deduplication related overheads were significantly reduced with adaptive block skipping, leading to a 30%-80% improvement in deduplication throughput when deduplieation mctadata were stored on the disk for data domain, and 25%-40% RAM space saving with a 15%-20% improvement in deduplication throughput when an in-RAM sparse index was used in sparse indexing. In both cases, the corresponding deduplication ratios reduced were below 5%.展开更多
文摘利用表格分析的方式,对比激光雷达系统(LIDAR)标准文件格式LAS version 2.0与旧版本verison1.1之间的异同,分析2.0版本的新特性,便于对LAS格式的理解和应用。新的定义标准更加适应LIDAR硬件间的通用性,给予用户更多的扩展空间,为LIDAR系统的应用提供高质量的文件交换基础。
基金This work is supported by the National Science Fund for Distinguished Young Scholars of China under Grant No. 61125102 and the Key Program of National Natural Science Foundation of China under Grant No. 61133008.
文摘We describe an efficient and easily applicable data deduplication framework with heuristic prediction based adaptive block skipping for the real-world dataset such as disk images to save deduplication related overheads and improve deduplication throughput with good deduplication efficiency maintained. Under the framework, deduplication operations are skipped for data chunks determined as likely non-duplicates via heuristic prediction, in conjunction with a hit and matching extension process for duplication identification within skipped blocks and a hysteresis mechanism based hash indexing process to update the hash indices for the re-encountered skipped chunks. For performance evaluation, the proposed framework was integrated and implemented in the existing data domain and sparse indexing deduplication algorithms. The experimental results based on a real-world dataset of 1.0 TB disk images showed that the deduplication related overheads were significantly reduced with adaptive block skipping, leading to a 30%-80% improvement in deduplication throughput when deduplieation mctadata were stored on the disk for data domain, and 25%-40% RAM space saving with a 15%-20% improvement in deduplication throughput when an in-RAM sparse index was used in sparse indexing. In both cases, the corresponding deduplication ratios reduced were below 5%.
