一种双三次插值实时超分辨率VLSI设计

A Real Time Super Resolution VLSI Design Based on Bicubic Interpolation

视频超分辨率技术具有广阔的应用前景,但基于深度学习方法的算法复杂度过高,难以实现实时计算.因此,近年来研究者们开始探索基于现场可编程逻辑门阵列(Field Programmable Gate Array,FPGA)的超分辨率算法加速器,以利用FPGA的优势来提高算法的性能和能耗,实现实时的视频超分辨率.设计了一种基于FPGA的高效高速双三次线性插值超大规模集成电路(Very Large Scale Integration Circuit,VLSI)架构,可用于4倍实时视频超分辨率.该FPGA架构解决了实现双三次插值过程中所需的复杂内存访问模式的问题,并提出了一种基于乒乓操作的数据重排硬件设计,将算法输出的特定顺序数据重新以行为主进行排列,使得硬件能够直接或较为简单地对接HDMI等视频接口.此外,采用状态机、流水线等方式降低设计功耗和减少时序违例,使得整个硬件设计可以更高频率运行.本研究在Zynq-7020 FPGA上实现了硬件架构,能够实时将qHD(960×540)的视频超采样为UHD(3840×2160)高清视频.实验结果表明,该硬件设计只需缓存1行图像像素,延迟仅为9.6μs,帧率达到192.9 Hz,成功实现实时处理.游戏图像数据集的测试结果表明,该设计峰值信噪比最高可达35.67 dB,结构相似度达到96.3%.

Video super-resolution technology has broad application prospects,but the algorithm complexity based on deep learning methods is too high to achieve real-time computation.In recent years,researchers have begun to explore super-resolution algorithm accelerators based on FPGA,in order to utilize the advantages of FPGA to improve algorithm performance and energy consumption,and achieve real-time video super-resolution.In this paper,an efficient high-speed bicubic linear interpolation VLSI architecture was designed based on FPGA,which can be used for 4×real-time video super-resolution.The FPGA architecture solved the problem of complex memory access mode required in the process of implementing bicubic interpolation,and proposed a hardware design of data rearrangement based on ping-pong operation,which rearranged the data in a specific order output by the algorithm into a row-major data arrangement,so that the hardware can directly connect to video interfaces such as HDMI.In addition,state machine,loop unrolling,pipeline and other methods were used to reduce design power consumption and timing violations,so that the entire hardware design can run at a higher frequency.The hardware architecture was implemented on Zynq-7020 FPGA,which enabled real-time oversampling of qHD(960×540)videos to UHD(3840×2160)high-definition videos.The experimental results show that the hardware design only needs to cache one row of image pixels with a latency of only 9.6μs and a frame rate of 192.9 Hz,and successfully achieves real-time processing.The test results on the game image dataset show that the design has a peak signal-to-noise ratio of up to 35.67 dB and a structural similarity of 96.3%.