显著性指导下图像迁移

Saliency-guided image translation

开拓了显著性指导下图像迁移新任务,在图像内容及保真度不变条件下,为使迁移后图像满足用户指定的显著图分布,提出一种全新的生成对抗网络(SalG-GAN)方法。对于给定的原始图像和目标显著图,所提方法可高效生成符合目标显著图的迁移图像。在所提方法中,引入解耦表示框架用于激励模型,针对相同的显著图输入,生成内容不同的迁移图像;在该框架基础上,设计基于显著图的注意模块作为一种特殊的注意力机制,辅助网络在图像迁移过程中聚焦于关键区域;同时,在所提方法中构造基于显著性的生成器、编码器、全局和局部鉴别器等深度网络结构。此外,建立用于显著性指导下图像迁移任务的大规模数据集,用于训练和评估所提方法,具体包括一个大规模的合成数据集和一个包括人眼视觉注意点的真实数据集。在2个数据集上的实验结果表明:所提方法在显著性指导下图像迁移任务中具有优异性能,远优于基准生成对抗网络方法。

This paper proposes a novel task for saliency-guided image translation,with the goal of image-to-image translation conditioned on the user specified saliency map.To address this problem,we develop a novel generative adversarial network(GAN)method-based model,called SalG-GAN method.Given the original image and target saliency map,proposed method can generate a translated image that satisfies the target saliency map.In proposed method,a disentangled representation framework is proposed to encourage the model to learn diverse translations for the same target saliency condition.A saliency-based attention module is introduced as a special attention mechanism to facilitate the developed structures of saliency-guided generator,saliency cue encoder,and saliency-guided global and local discriminators.Furthermore,we build a synthetic dataset and a real-world dataset with labeled visual attention for training and evaluating proposed method.The experimental results on both datasets verify the effectiveness of our model for saliency-guided image translation.