基于子空间特征相互学习的MRI与PET/SPECT图像融合

MRI and PET/SPECT Image Fusion Based on Subspace Feature Mutual Learning

在医学图像中,MRI图像提供包含细节的纹理结构信息和较好的分辨率,而PET/SPECT图像保留了分子活性信息以及颜色功能信息,因此,将它们进行融合是一项重要的任务。大部分现有的方法在融合过程中存在颜色失真、模糊和噪声等问题。为此,提出了一种新的基于子空间注意力孪生自编码网络(Subspace Attention-Siamese Auto-encoding Network,SSASAEN)来融合MRI和PET/SPECT图像中有意义的信息。在图像融合网络中提出SSA-SAEN,引入了子空间特征相互学习概念,利用子空间注意力模块,使MRI和PET/SPECT图像能够在学习自己特征的同时互相学习彼此的特征,同时减少信息冗余,保证高效、完整的特征提取。此外,通过条件概率模型对所提取的特征进行互补融合,同时将加权保真项、梯度损失项加入到训练网络中,以达到网络优化的目的。在公共数据集上进行的大量定性和定量实验表明,该方法能够得到一幅清晰的融合图像,表明了该方法与其他先进方法相比的优越性和有效性。

In medical imaging,MRI images provide detailed texture information and better resolution,while PET/SPECT images retain molecular activity information and color function information,so fusing them is an important task.Most of the existing methods have some problems in the fusion process,such as color distortion,blur and noise.Therefore,a new subspace attentionsiamese auto-encoding network(SSA-SAEN)is proposed to fully fuse meaningful information from MRI and PET/SPECT images.SSA-SAEN is proposed in image fusion network,and the subspace feature mutual learning concept is introduced.By using subspace attention module,MRI and PET/SPECT images can learn each other’s features,while reducing information redundancy and ensuring efficient and complete feature extraction.In addition,the conditional probability model is used to complement and fuse the extracted features,and the weighted fidelity gradient loss term is added into the training network to achieve the goal of network optimization.A large number of qualitative and quantitative experiments on public datasets show that the proposed method can obtain a clear fused image,which demonstrates the superiority and effectiveness of the proposed method compared with other advanced methods.