联合语义边缘与有向全变分的纹理-结构分解

Texture-Structure Decomposition with Semantic Edge Detection and Directional Total Variation

在去除尺度变化较大的纹理时,基于低级视觉特征的纹理-结构分解方法难以准确保留语义上重要的弱边缘,针对该问题提出了融合语义边缘检测与有向全变分模型的纹理-结构分解方法.首先在不同尺度下利用多卷积特征网络估计各像素的语义边缘置信度;然后在纹理的局部振荡假设基础上以有向全变分估计各像素的纹理度,为进一步有效地区分纹理边缘和结构边缘,引入块平移算法修正纹理度,并结合边缘置信度优化分解模型的权重.在BSDS500, NYUD以及RTVD数据集上的实验结果表明,该方法在分解准确性和视觉质量上优于现有多种方法,且易于通过GPU加速高分辨率图像的分解效率.

With filters based on low-level vision, it is challenging to remove multi-scale texture accurately and avoid overblurred weak edges in the texture-structure decomposition. Aiming at the problem, this paper proposes a method which combines semantic edge detection with directional total variation for the decomposition task. Firstly, a network with richer convolutional features is used to extract multi-scale edges of different objects and estimates edge confidence for each pixel. Then, texture confidence of each pixel is estimated from the directional total variation, and the patch shifting algorithm is used to smooth out the strong texture edges which are often misclassified as structure. Finally, the edge confidences are incorporated into texture confidence to weight the decomposition model. Experiments are conducted on the three datasets, which are BSDS500, NYUD and RTVD, and the results demonstrate our method produces more accurate structure layers and provides better visual quality compared to state-of-the-art of decomposition methods. The GPU-based implementation of our method is fast even for high-resolution images.