基于多模态特征融合的脑瘤图像分割方法

Brain tumor image segmentation method based on multi-modal feature fusion

针对目前大多数医学图像分割方法难以对多模态图像进行特征融合进而完成精确分割任务的问题,提出一种基于编码器-解码器总体架构的多模态脑瘤图像特征融合策略。首先,编码阶段利用孪生网络对不同模态数据进行特征提取,孪生网络结构参数和权值共享的特性可有效减少网络参数量;其次,在进行特征提取的编码阶段加入级间融合,保留不同模态的共性特征的同时强调其互补特征;然后,在解码阶段引入密集跳跃连接思想,最大程度结合不同尺度特征图的低级细节和高级语义信息;最后,设计混合损失函数,在网络生成的预测图受真值图监督的同时让最高级特征融合图也受同倍下采样真值图的监督。所提方法在公开数据集BraTS2019上进行实验,并用图像分割常用的5种指标进行评估。在脑瘤及水肿区域分割任务中得到平均Dice系数为0.884,阳性预测率为0.870,灵敏度为0.898,豪斯多夫距离为3.917,平均交并比达到79.1%,与较先进的算法U-Net和PA-Net相比多项指标均有提升。实验结果说明,级间融合和层间跳跃连接的加入对多模态医学图像的分割效果有所提升,在医学上对脑肿瘤磁共振图像进行病变区域分割具有重要的应用价值和理论意义。

Aiming at the problem that most of the current medical image segmentation methods are difficult to perform feature fusion for multi-modal images to achieve accurate segmentation, a multi-modal brain tumor image feature fusion strategy based on encoder and decoder overall architecture is proposed. In the coding phase, twin networks are used to extract features from different modal data. The number of network parameters can be effectively reduced by sharing the structural parameters and weights of twin networks. In addition, the interstage fusion is added in the coding phase of feature extraction to keep the common features of different modes while emphasizing their complementary features. Then, the idea of dense skip connection is introduced in the decoding phase to maximize the combination of low-level details and high-level semantic information of feature maps at different scales. Finally, a mixed loss function is designed, so that the prediction graph generated by the network is supervised by the truth graph, and that the highest-level feature fusion graph is also supervised by the truth graph sampled under the same multiplier. The proposed method is tested on the public data set BraTS2019 and evaluated with 5 commonly used indexes for image segmentation. For the segmentations of brain tumor and edema area, the proposed method are superior to more advanced algorithms U-NET and PA-NET in many indexes, and the average Dice coefficient, positive prediction rate, sensitivity, Hausdorff distance and mean intersection over union of the proposed method are 0.884, 0.870, 0.898, 3.917 and 79.1%, respectively. The experimental results reveal that the addition of interstage fusion and interlayer skip connection can improve the segmentation performance of multimodal medical images, and it has important application value and theoretical significance in the segmentation of brain tumor in magnetic resonance image.