Tongue image segmentation and tongue color classification based on deep learning

基于深度学习的舌体图像分割和舌色分类研究

Objective To propose two novel methods based on deep learning for computer-aided tongue diagnosis,including tongue image segmentation and tongue color classification,improving their diagnostic accuracy.Methods LabelMe was used to label the tongue mask and Snake model to optimize the labeling results.A new dataset was constructed for tongue image segmentation.Tongue color was marked to build a classified dataset for network training.In this research,the Inception+Atrous Spatial Pyramid Pooling(ASPP)+UNet(IAUNet)method was proposed for tongue image segmentation,based on the existing UNet,Inception,and atrous convolution.Moreover,the Tongue Color Classification Net(TCCNet)was constructed with reference to ResNet,Inception,and Triple-Loss.Several important measurement indexes were selected to evaluate and compare the effects of the novel and existing methods for tongue segmentation and tongue color classification.IAUNet was compared with existing mainstream methods such as UNet and DeepLabV3+for tongue segmentation.TCCNet for tongue color classification was compared with VGG16 and GoogLeNet.Results IAUNet can accurately segment the tongue from original images.The results showed that the Mean Intersection over Union(MIoU)of IAUNet reached 96.30%,and its Mean Pixel Accuracy(MPA),mean Average Precision(mAP),F1-Score,G-Score,and Area Under Curve(AUC)reached 97.86%,99.18%,96.71%,96.82%,and 99.71%,respectively,suggesting IAUNet produced better segmentation than other methods,with fewer parameters.Triplet-Loss was applied in the proposed TCCNet to separate different embedded colors.The experiment yielded ideal results,with F1-Score and mAP of the TCCNet reached 88.86% and 93.49%,respectively.Conclusion IAUNet based on deep learning for tongue segmentation is better than traditional ones.IAUNet can not only produce ideal tongue segmentation,but have better effects than those of PSPNet,SegNet,UNet,and DeepLabV3+,the traditional networks.As for tongue color classification,the proposed network,TCCNet,had better F1-Score and mAP values as compared with other neural networks such as VGG16 and GoogLeNet.

目的为了提高计算机辅助舌诊的准确率,提出了两种基于深度学习的新方法,分别用于舌体图像分割和舌色分类。方法利用LabelMe对舌体掩码进行标注,并利用Snake模型优化标注结果,构建新的舌体分割数据集。通过标注舌色为之后的网络训练构建舌象分类数据集。在本研究中,结合现有的UNet、Inception和空洞卷积提出用于舌体分割的Inception+空洞卷积空间金字塔池化(ASPP)+UNet(IAUNet)。此外,参考Res-Net、Inception和Triplet-Loss构建用于舌色分类网络的Tongue Color Classification Net(TCCNet)。选取一系列重要的度量因子用于评估和比较新的和现有的舌体分割方法和舌色分类方法的效果。针对舌体分割使用IAUNet与UNet、DeepLabV3+等现有主流方法进行对比实验;针对舌色分类使用TCCNet与VGG16和GoogLeNet等进行对比实验。结果IAUNet能够精确地从原始图像中分割出舌体。结果表明,IAUNet的平均交并比(MIoU)达到了96.30%,平均像素精度(MPA)、平均精度均值(mAP)、F1-Score、G-Score和曲线下面积(AUC)值分别达到97.86%、99.18%、96.71%、96.82%和99.71%,表明IAUNet的分割效果优于其他方法,且所用参数更少。TCCNet在舌色分类中引入Triplet-Loss并用于将不同类的嵌入分离,取得了理想的实验效果,TCCNet的F1-Score和mAP分别达到88.86%和93.49%。结论基于深度学习的舌体分割方法IAUNet优于传统方法,能够实现较为理想的舌体分割,分割效果优于PSPNet、SegNet、UNet和DeepLabV3+。舌色分类网络TCCNet相较于其他分类神经网络如VGG16和GoogLeNet,在F1-Score和mAP等指标上都表现得更为优异。