基于目标图像先验信息的无监督多聚焦图像融合

Unsupervised multi-focus image fusion based on target image prior information

多聚焦图像融合(MFIF)是从不同源图像中获取聚焦区域,以形成全清晰图像的一种图像增强方法。针对目前MFIF方法主要存在的两个方面问题,即传统的空间域方法在其融合边界存在较强的散焦扩散效应(DSE)以及伪影等问题;深度学习方法缺乏还原光场相机生成的数据集,并且因需要大量手动调参而存在训练过程耗时过多等问题,提出了一种基于目标图像先验信息的无监督多聚焦图像融合方法。首先,将源图像本身的内部先验信息和由空间域方法生成的初始融合图像所具有的外部先验信息分别用于G-Net和FNet输入,其中,G-Net和F-Net都是由U-Net组成的深度图像先验(DIP)网络;然后,引入一种由空间域方法生成的参考掩膜辅助G-Net生成引导决策图;最后,该决策图联合初始融合图像对F-Net进行优化,并生成最终的融合图像。验证实验基于具有真实参考图像的Lytro数据集和融合边界具有强DSE的MFFW数据集,并选用了5个广泛应用的客观指标进行性能评价。实验结果表明,该方法有效地减少了优化迭代次数,在主观和客观性能评价上优于8种目前最先进的MFIF方法,尤其在融合边界具有强DSE的数据集上表现得更有优势。

Multi-focus image fusion(MFIF)is an image enhancement method that combines the focused regions from different source images to form a fully sharp image.Currently,in the context of MFIF methods,there are two main challenges.First,traditional methods such as spatial domain approaches produce fusion images with high objective scores,but they suffer from strong defocus spread effects(DSE)and artifacts at the fusion boundaries.Second,deep learning methods lack a dataset generated from plenoptic cameras and require extensive manual parameter tuning,resulting in time-consuming training processes.To address these challenges,this paper proposed an unsupervised multi-focus image fusion method based on target image prior information.Firstly,it utilized the internal prior information of the source image itself and the external prior information of the initial fusion image generated by a spatial domain method as inputs for the G-Net and F-Net,respectively,both the G-Net and F-Net were components of the UNet-based deep image prior(DIP)network.Then,it introduced a reference mask generated by a spatial domain method to assist G-Net network for generating a guiding decision map.Finally,it used the decision map and the initial fusion image to jointly optimize the F-Net,producing the final fusion image.It conducted validation experiments on the Lytro dataset with real reference images and the MFFW dataset with strong DSE exhibiting in the fusion boundaries,and employed five widely used objective metrics for performance evaluation.The experimental results demonstrate that the proposed method significantly reduces the number of optimization iterations,and outperforms eight state-of-the-art MFIF approaches in terms of the subjective and objective performance evaluation,and especially shows superior performance on the datasets with strong DSE exhibiting in the fusion boundaries.