基于深度学习的两阶段多假设视频压缩感知重构算法

Two-Stage Multi-Hypothesis Network for Compressed Video Sensing Reconstruction Algorithms Based on Deep Learning

传统视频压缩感知重构算法重构时延过长,新发展的基于神经网络的视频压缩感知重构算法虽解决了高耗时的问题但未能充分利用视频的时空相关性,重构质量较差。为了解决上述问题,文中提出了基于深度学习的两阶段多假设视频压缩感知重构算法(2sMHNet)。首先,采用时域可变形卷积对齐网络实现基于像素的深度学习多假设预测,在避免了块效应的同时通过自适应参数学习提高了假设集的匹配准确性与权重的计算精度,充分地挖掘了时间相关性得到高质量的预测帧;然后,构建残差重构模块以实现预测帧残差的观测域重构,进一步提升重构质量;最后,为了充分利用图像组帧间相关性,提出了两阶段串行式重构模式,在第一阶段利用细节信息丰富的关键帧提升非关键帧重构质量,在第二阶段利用相关性更强的相邻帧再次进行运动补偿重构,适应运动快且复杂的序列。仿真结果证明,2sMHNet相比于目前优秀的视频压缩感知重构算法具有更加优良的重构性能。

Traditional Compressed Video Sensing(CVS)reconstruction algorithm is highly time-consuming.Newly developed CVS neural networks can successfully deal with the speed problem,but it fails to make full use of the spatiotemporal correlation of video and leads to a poor performance.To solve this problem,a novel two-stage multi-hypothesis neural network(2sMHNet)was proposed.Firstly,the Temporal Deformable Alignment Network(TDAN)was used to realize pixel based multi-hypothesis prediction.While avoiding block effects,it improves the matching accuracy of the hypothesis set and obtains accurate multi-hypothesis weights by adaptively parameters learning.Then,the residual reconstruction module was constructed to reconstruct the prediction residual with measurements to further improve the reconstruction quality.Finally,in order to make full use of the inter-frame correlation,a two-stage serial reconstruction mode was proposed.In the first stage,as the reconstructed key frames have rich details,they are selected as the reference frame to improve the non-key frames quality.In the second stage,the more relevant adjacent frames are used for motion compensation,which is more conducive to fast and complex sequences.Experimental results demonstrate that the proposed 2sMHNet outperforms the existing good CVS reconstruction algorithms.