基于多模态融合和自适应剪枝Transformer的脑肿瘤图像分割算法

Brain Tumor Segmentation Algorithm Based on Multi-modal Fusion and Adaptive Pruning Transformer

脑肿瘤是目前世界上最致命的肿瘤之一,所以脑肿瘤图像的自动分割在临床诊疗中变得日益重要.近年来,基于CNN和Transformer的脑肿瘤分割方法在医学图像分割领域取得了令人欣喜的成就.然而,大多数方法没有充分利用脑肿瘤多模态间的互补性和差异性,并且模型中的Transformer在捕获远程依赖性的同时,忽略了其较大的计算复杂性、冗余依赖性等问题.针对此问题,提出一种基于多模态融合和自适应剪枝Transformer的脑肿瘤图像分割方法(MF-MAPT Swin UNETR),其中多模态融合模块可以充分学习性质相近的模态间信息和不同模态不同尺度的特征变化,为后续分割提供了充分的准备;基于多模态的自适应剪枝Transformer可以降低计算复杂度,对提升性能有一定的帮助,将MF-MAPT Swin UNETR模型在两个公共数据集上进行了实验验证,结果表明,该模型较最先进的方法整体具有突出的分割性能.

Brain tumors are one of the most lethal tumors in the world,so automatic segmentation of brain tumor images is becoming increasingly important in clinical diagnosis and treatment.In recent years,brain tumor segmentation methods based on CNN and Transformer have achieved gratifying achievements in the field of medical image segmentation.However,in most methods the complementarity and difference between brain tumor multimodalities are not fully exploited,and in methods with Transformer where long-range dependence is captured,the large computational complexity and redundant dependencies are unbearable.To solve the two problems,a brain tumor image segmentation method(MF-MAPT Swin UNETR)based on multi-modal fusion and adaptive pruning Transformer is proposed.The multi-modal fusion module can fully learn the characteristics between modalities with similar properties and feature changes in different modes and scales provide sufficient preparation for subsequent segmentation;the multi-modal adaptive pruning Transformer can reduce com-putational complexity and help improve performance.The model is experimentally verified on two public datasets,and it is shown by the experimental results that the proposed MF-MAPT Swin UNETR model exhibits outstanding segmentation performance overall compared with state-of-the-art methods.