We describe an efficient and easily applicable data deduplication framework with heuristic prediction based adaptive block skipping for the real-world dataset such as disk images to save deduplication related overhead...We describe an efficient and easily applicable data deduplication framework with heuristic prediction based adaptive block skipping for the real-world dataset such as disk images to save deduplication related overheads and improve deduplication throughput with good deduplication efficiency maintained. Under the framework, deduplication operations are skipped for data chunks determined as likely non-duplicates via heuristic prediction, in conjunction with a hit and matching extension process for duplication identification within skipped blocks and a hysteresis mechanism based hash indexing process to update the hash indices for the re-encountered skipped chunks. For performance evaluation, the proposed framework was integrated and implemented in the existing data domain and sparse indexing deduplication algorithms. The experimental results based on a real-world dataset of 1.0 TB disk images showed that the deduplication related overheads were significantly reduced with adaptive block skipping, leading to a 30%-80% improvement in deduplication throughput when deduplieation mctadata were stored on the disk for data domain, and 25%-40% RAM space saving with a 15%-20% improvement in deduplication throughput when an in-RAM sparse index was used in sparse indexing. In both cases, the corresponding deduplication ratios reduced were below 5%.展开更多
适用于高效视频编码(High Efficiency Video Coding,HEVC)视频的数据隐藏方案相对较少,现有方案大多无法充分利用所有类型视频帧,并且存在嵌入容量不高或载密视频流比特率增加较大等问题.文中提出了一种HEVC视频数据隐藏方法,利用HEVC...适用于高效视频编码(High Efficiency Video Coding,HEVC)视频的数据隐藏方案相对较少,现有方案大多无法充分利用所有类型视频帧,并且存在嵌入容量不高或载密视频流比特率增加较大等问题.文中提出了一种HEVC视频数据隐藏方法,利用HEVC视频新的编码元素实现在不同类型视频帧中的数据嵌入.该方法主要包括三种嵌入模式:修改编码树单元的样值自适应补偿值(Sample Adaptive Offset,SAO)、交换16×16、8×8和4×4编码单元中的残差系数以及改变4×4变换跳过块的符号位.实验结果表明,所提出的方法具有较好的嵌入不可感知性和较高的嵌入容量,并对视频流比特率影响很小.与最近提出的方案相比,载密视频具有更高的视觉质量和更小的比特率增加.展开更多
基金This work is supported by the National Science Fund for Distinguished Young Scholars of China under Grant No. 61125102 and the Key Program of National Natural Science Foundation of China under Grant No. 61133008.
文摘We describe an efficient and easily applicable data deduplication framework with heuristic prediction based adaptive block skipping for the real-world dataset such as disk images to save deduplication related overheads and improve deduplication throughput with good deduplication efficiency maintained. Under the framework, deduplication operations are skipped for data chunks determined as likely non-duplicates via heuristic prediction, in conjunction with a hit and matching extension process for duplication identification within skipped blocks and a hysteresis mechanism based hash indexing process to update the hash indices for the re-encountered skipped chunks. For performance evaluation, the proposed framework was integrated and implemented in the existing data domain and sparse indexing deduplication algorithms. The experimental results based on a real-world dataset of 1.0 TB disk images showed that the deduplication related overheads were significantly reduced with adaptive block skipping, leading to a 30%-80% improvement in deduplication throughput when deduplieation mctadata were stored on the disk for data domain, and 25%-40% RAM space saving with a 15%-20% improvement in deduplication throughput when an in-RAM sparse index was used in sparse indexing. In both cases, the corresponding deduplication ratios reduced were below 5%.
