基于OpenCL的Gzip数据压缩算法

OpenCL-based optimization for Gzip algorithm

目前,数据压缩算法主要基于串行编程模型设计和实现,导致数据压缩速率较低,因而无法满足大数据分析和处理等应用的实时性需求。为了解决这个问题,以常用数据压缩算法Gzip为蓝本,提出一种实现无损数据压缩算法的专用硬件电路。首先,采用多字典并行查找的设计方案提高重复数据的查找速率;接着,采用匹配长度拼接技术提升数据的压缩效果;最后,采用开放计算语言(Open CL)实现了所提出的专用硬件电路。基于现场可编程门阵列(FPGA)进行功能验证和性能评测,结果表明:与基于串行编程模型设计和实现的数据压缩算法相比,所提出的硬件电路在取得适当压缩率的同时,显著地提高了数据的压缩速率,压缩速率可达12 Gb/s。

Data compression algorithms are usually designed and implemented by using the sequential programming model, resulting in a very low data compression speed when they are deployed on general-purpose processors, which cannot satisfy the real-time requirement of applications such as big data analytics. To address this problem, a specialized lossless data compression circuit that implemented the Gzip algorithm was proposed. First of all, a parallelized lookup structure with multiple dictionaries was designed to increase the speed of searching for repeated data. Then, a match length splicing technique was proposed to improve the compression ratio. Finally, the circuit was implemented by using Open Computing Language （OpeuCL）. Results verified on the Field-Programming Gate Array （FPGA） show that the proposed circuit can achieve moderate compression ratio and dramatically improve the data compression speed, which can be as high as 12 Gb/s.