融合CNN和Transformer的并行双分支皮肤病灶图像分割

Parallel dual-branch image segmentation of skinlesions fusing CNN and Transformer

准确的皮肤病变自动分割对于协助医生临床诊断和治疗至关重要。针对现有卷积结构能提取局部特征信息但无法建模长程依赖关系,而Transformer能提取全局上下文信息但存在细节信息丢失的问题,提出了一种融合CNN和Transformer的并行多尺度自动分割网络PDTransCNN。首先以基于ResNet34的CNN分支和Transformer分支并行提取皮肤病图像的特征信息,构建多级局部相关性和捕获上下文信息间的长距离依赖关系;其次利用特征融合模块FM互补两分支特征间的关键信息,增强语义信息间的依赖关系;最后采用Transformer解码单元逐步融合编码块和融合单元所提取到的语义信息得到最终分割结果。该模型在ISIC2016、ISIC2017和ISIC2018数据集上进行测试,其Dice系数分别高达91.72%、87.34%和90.01%,IoU值分别为85.6%、79.55%和83.67%。实验结果表明,PDTransCNN相比其他分割模型具有更好的分割性能,能清晰有效地分割皮肤病变图像。

Accurate automatic segmentation of skin lesions is crucial to assist physicians in clinical diagnosis and treatment.Aiming at the problem that convolutional structure can extract local feature information but cannot model long-range dependencies,while Transformer can extract global context information but suffers from the loss of detail information,this paper proposed a parallel multi-scale automatic segmentation network PDTransCNN that integrated CNN and Transformer.Firstly,it constructed multi-level local correlation and captured long-range dependencies between contextual information by extracting the feature information of dermatological images in parallel with the CNN branch based on ResNet34 and Transformer branch.Secondly,it utilized the feature fusion module(FM)to complement the key information between the two branches of features and enhance the dependencies of the semantic information.Finally,it used the Transformer decoding unit to gradually fuse the semantic information extracted from the encoding block and the fusion unit in order to obtain the final segmentation result.The model was tested on ISIC2016,ISIC2017 and ISIC2018 datasets with Dice coefficients as high as 91.72%,87.34%and 90.01%,and IoU values of 85.60%,79.55%and 83.67%,respectively.The experimental results show that PDTransCNN has better segmentation performance compared to other segmentation models and can segment skin lesion images clearly and effectively.