基于3层分解和卷积神经网络的多模态医学图像融合

Multi-modal Medical Image Fusion Based on Three-layer Decomposition and Convolutional Neural Network

针对多尺度分解(multi-scale decomposition,MSD)方法会导致图像细微组织结构丢失和产生噪声的问题,结合结构纹理分解模型、改进的卷积神经网络和相位一致性方法,提出1种新的多模态医学图像融合方法。首先,针对MSD容易产生噪声的问题,引入低通滤波器优化函数和结构纹理分解模型,有效解决了分解技术的难题;其次,对结构纹理部分设计1种基于卷积神经网络结合高斯平滑的融合方法,加强了对图像细节部分的提取,消除了噪声,并对高频部分引入基于相位一致性的改进方法进行融合;然后通过分解过程的逆变换得到最终的融合图像。定性、定量分析表明,所提出的算法在视觉效果、互信息(mutual information,MI)、特征互信息(feature mutual information,FMI)、结构相似性(structure similarity index measure,SSIM)、信息熵(entropy of information,EN)、峰值信噪比(peak signal to noise ration,PSNR)方面都达到了较高水平,融合后的图像更有利于专家和医生的诊断。

The multi-scale decomposition(MSD)method will lead to loss of the fine organizational structure of the image and generation of noise.A new multi-modal medical image fusion method is proposed by combining the structural texture decomposition model,improved convolutional neural network and phase consistency method.Firstly,aiming at the phenomenon that MSD is liable to produce noise,low pass filter optimization function and structure texture decomposition model are introduced to effectively solve the problem of decomposition technology.Secondly,a fusion method based on convolutional neural network combined with Gaussian smoothing is designed for the structural texture part,which enhances the extraction of details and eliminated the noise.The high-frequency part is fused with an improved method based on phase consistency.The final fusion image is obtained by the inverse transformation of the decomposition process.Through qualitative and quantitative analysis,the proposed algorithm has achieved a higher level in terms of visual effect,mutual information(MI),feature mutual information(FMI),structure similarity index measure(SSIM)and entropy of information(EN),peak signal to noise ration(PSNR)and the fused image is more conducive to the diagnosis of experts and doctors.