一种多尺度残差注意力Unet-Like网络的医学图像融合方法

Multiscale Residual Attention Unet-Like Network for Medical Image Fusion

针对目前医学图像融合任务中融合策略难以制定及模态重要信息挖掘不够的问题,提出了一种多尺度残差注意力Unet-Like网络(multiscale residual attention unet-like network, MRAUnet)框架用于医学图像融合。首先,传统的融合算法在处理多模态医学图像时存在特征学习能力不足的问题,无法更好地保留互补的多模信息。因此,为了增强模型的特征提取能力与稳定性,MRAUnet将残差注意力机制注入到特征提取过程中。其次,Unet架构在理解医学图像的语义上有着天然的优势,而且它的U型结构使得Unet对浅层简单特征与深层抽象特征有更强的感知能力和重用能力,所以MRAUnet采用了Unet架构来增强模型的语义感知能力和特征重用,同时整体网络采用多尺度的方式进行特征处理以强化Unet架构的特征抽取能力。另外,为了丰富在融合图像中的模态信息,还设计了一种基于结构信息与细节信息保留的损失函数用于训练MRAUnet。最后,MRAUnet是一个端到端的融合网络,无需人工制定融合策略,可以有效地利用多模态图像的浅层结构特征与深层抽象特征,以平衡融合图像中的解剖信息和功能信息,在保持较高清晰度的同时可以显示出元素放射量与血液流动性等功能特征。在哈佛医学院全脑图集上的实验结果表明,MRAUnet与近期较为先进的算法相比,在主观和客观评判2个方面都取得了较大进步,在基于人类视觉感知的保真度、基于特征保留的互信息等指标上均有较大优势。

Aiming at the difficulties of formulating fusion strategies and incomplete preservation of modal information in medical image fusion tasks,a multi-scale residual attention Unet-Like network is proposed for medical image fusion,namely MRAUnet(Multiscale Residual Attention Unet-like Network).Firstly,most traditional fusion algorithms cannot extract sufficient complementary information from multimodal medical images.To enhance the feature extraction ability and stability of the model,MRAUnet employs residual attention mechanism in the feature extraction processing.Secondly,the Unet structure has great advantages in the semantic perception of medical images,and its U-shaped structure enables Unet to have stronger perception and reuse capabilities for shallow simple features and deep abstract features.Therefore,MRAUnet adopts the Unet architecture to enhance the semantic perception ability and feature reuse of the model,while the overall network adopts a multi-scale fashion for feature processing to enhance the feature extraction ability of the Unet architecture.In addition,to capture more modal information,a loss function based on structural information and detail information retention is designed to train MRAUnet.Finally,MRAUnet is an end-to-end fusion network with no need to manually design fusion strategies.The proposed network can effectively capture the shallow structural features and deep abstract features from the source image,effectively balancing the anatomical and functional information in the fused image,while maintaining high clarity and displaying functional features such as element radiation and blood flow.The experimental results on the Harvard Medical School Whole Brain Atlas dataset indicate that the MRAUnet has significant improvements in both subjective and objective evaluation compared to some recent advanced algorithms,which has great advantages in terms of VIF based on human visual fidelity,and MI based on feature retention and other metrics.