Objective In tongue diagnosis,the location,color,and distribution of spots can be used to speculate on the viscera and severity of the heat evil.This work focuses on the image analysis method of artificial intelligenc...Objective In tongue diagnosis,the location,color,and distribution of spots can be used to speculate on the viscera and severity of the heat evil.This work focuses on the image analysis method of artificial intelligence(AI)to study the spotted tongue recognition of traditional Chinese medicine(TCM).Methods A model of spotted tongue recognition and extraction is designed,which is based on the principle of image deep learning and instance segmentation.This model includes multiscale feature map generation,region proposal searching,and target region recognition.Firstly,deep convolution network is used to build multiscale low-and high-abstraction feature maps after which,target candidate box generation algorithm and selection strategy are used to select high-quality target candidate regions.Finally,classification network is used for classifying target regions and calculating target region pixels.As a result,the region segmentation of spotted tongue is obtained.Under non-standard illumination conditions,various tongue images were taken by mobile phones,and experiments were conducted.Results The spotted tongue recognition achieved an area under curve(AUC)of 92.40%,an accuracy of 84.30%with a sensitivity of 88.20%,a specificity of 94.19%,a recall of 88.20%,a regional pixel accuracy index pixel accuracy(PA)of 73.00%,a mean pixel accuracy(m PA)of73.00%,an intersection over union(Io U)of 60.00%,and a mean intersection over union(mIo U)of 56.00%.Conclusion The results of the study verify that the model is suitable for the application of the TCM tongue diagnosis system.Spotted tongue recognition via multiscale convolutional neural network(CNN)would help to improve spot classification and the accurate extraction of pixels of spot area as well as provide a practical method for intelligent tongue diagnosis of TCM.展开更多
Objective To propose two novel methods based on deep learning for computer-aided tongue diagnosis,including tongue image segmentation and tongue color classification,improving their diagnostic accuracy.Methods LabelMe...Objective To propose two novel methods based on deep learning for computer-aided tongue diagnosis,including tongue image segmentation and tongue color classification,improving their diagnostic accuracy.Methods LabelMe was used to label the tongue mask and Snake model to optimize the labeling results.A new dataset was constructed for tongue image segmentation.Tongue color was marked to build a classified dataset for network training.In this research,the Inception+Atrous Spatial Pyramid Pooling(ASPP)+UNet(IAUNet)method was proposed for tongue image segmentation,based on the existing UNet,Inception,and atrous convolution.Moreover,the Tongue Color Classification Net(TCCNet)was constructed with reference to ResNet,Inception,and Triple-Loss.Several important measurement indexes were selected to evaluate and compare the effects of the novel and existing methods for tongue segmentation and tongue color classification.IAUNet was compared with existing mainstream methods such as UNet and DeepLabV3+for tongue segmentation.TCCNet for tongue color classification was compared with VGG16 and GoogLeNet.Results IAUNet can accurately segment the tongue from original images.The results showed that the Mean Intersection over Union(MIoU)of IAUNet reached 96.30%,and its Mean Pixel Accuracy(MPA),mean Average Precision(mAP),F1-Score,G-Score,and Area Under Curve(AUC)reached 97.86%,99.18%,96.71%,96.82%,and 99.71%,respectively,suggesting IAUNet produced better segmentation than other methods,with fewer parameters.Triplet-Loss was applied in the proposed TCCNet to separate different embedded colors.The experiment yielded ideal results,with F1-Score and mAP of the TCCNet reached 88.86% and 93.49%,respectively.Conclusion IAUNet based on deep learning for tongue segmentation is better than traditional ones.IAUNet can not only produce ideal tongue segmentation,but have better effects than those of PSPNet,SegNet,UNet,and DeepLabV3+,the traditional networks.As for tongue color classification,the proposed network,TCCNet,had better F1-Score and mAP values as compared with other neural networks such as VGG16 and GoogLeNet.展开更多
基金Anhui Province College Natural Science Fund Key Project of China(KJ2020ZD77)the Project of Education Department of Anhui Province(KJ2020A0379)。
文摘Objective In tongue diagnosis,the location,color,and distribution of spots can be used to speculate on the viscera and severity of the heat evil.This work focuses on the image analysis method of artificial intelligence(AI)to study the spotted tongue recognition of traditional Chinese medicine(TCM).Methods A model of spotted tongue recognition and extraction is designed,which is based on the principle of image deep learning and instance segmentation.This model includes multiscale feature map generation,region proposal searching,and target region recognition.Firstly,deep convolution network is used to build multiscale low-and high-abstraction feature maps after which,target candidate box generation algorithm and selection strategy are used to select high-quality target candidate regions.Finally,classification network is used for classifying target regions and calculating target region pixels.As a result,the region segmentation of spotted tongue is obtained.Under non-standard illumination conditions,various tongue images were taken by mobile phones,and experiments were conducted.Results The spotted tongue recognition achieved an area under curve(AUC)of 92.40%,an accuracy of 84.30%with a sensitivity of 88.20%,a specificity of 94.19%,a recall of 88.20%,a regional pixel accuracy index pixel accuracy(PA)of 73.00%,a mean pixel accuracy(m PA)of73.00%,an intersection over union(Io U)of 60.00%,and a mean intersection over union(mIo U)of 56.00%.Conclusion The results of the study verify that the model is suitable for the application of the TCM tongue diagnosis system.Spotted tongue recognition via multiscale convolutional neural network(CNN)would help to improve spot classification and the accurate extraction of pixels of spot area as well as provide a practical method for intelligent tongue diagnosis of TCM.
基金Scientific Research Project of the Education Department of Hunan Province(20C1435)Open Fund Project for Computer Science and Technology of Hunan University of Chinese Medicine(2018JK05).
文摘Objective To propose two novel methods based on deep learning for computer-aided tongue diagnosis,including tongue image segmentation and tongue color classification,improving their diagnostic accuracy.Methods LabelMe was used to label the tongue mask and Snake model to optimize the labeling results.A new dataset was constructed for tongue image segmentation.Tongue color was marked to build a classified dataset for network training.In this research,the Inception+Atrous Spatial Pyramid Pooling(ASPP)+UNet(IAUNet)method was proposed for tongue image segmentation,based on the existing UNet,Inception,and atrous convolution.Moreover,the Tongue Color Classification Net(TCCNet)was constructed with reference to ResNet,Inception,and Triple-Loss.Several important measurement indexes were selected to evaluate and compare the effects of the novel and existing methods for tongue segmentation and tongue color classification.IAUNet was compared with existing mainstream methods such as UNet and DeepLabV3+for tongue segmentation.TCCNet for tongue color classification was compared with VGG16 and GoogLeNet.Results IAUNet can accurately segment the tongue from original images.The results showed that the Mean Intersection over Union(MIoU)of IAUNet reached 96.30%,and its Mean Pixel Accuracy(MPA),mean Average Precision(mAP),F1-Score,G-Score,and Area Under Curve(AUC)reached 97.86%,99.18%,96.71%,96.82%,and 99.71%,respectively,suggesting IAUNet produced better segmentation than other methods,with fewer parameters.Triplet-Loss was applied in the proposed TCCNet to separate different embedded colors.The experiment yielded ideal results,with F1-Score and mAP of the TCCNet reached 88.86% and 93.49%,respectively.Conclusion IAUNet based on deep learning for tongue segmentation is better than traditional ones.IAUNet can not only produce ideal tongue segmentation,but have better effects than those of PSPNet,SegNet,UNet,and DeepLabV3+,the traditional networks.As for tongue color classification,the proposed network,TCCNet,had better F1-Score and mAP values as compared with other neural networks such as VGG16 and GoogLeNet.
