基于多模态融合的肝脏纤维化自动分类的算法

An Algorithm for Automatic Classification of Liver Fibrosis Based on Multi-modality Fusion

针对传统临床检查中血清生化物指标诊断精度较低,以及视觉评估磁共振图像无法精确对肝脏纤维化程度分级的缺点,设计了一种肝脏纤维化分级的多模态多通道融合的深度学习算法(MCMD-Resnet18)。该算法充分利用磁共振图像数据和临床文本数据二者之间的语义信息,在网络的分类器阶段将图像文本两个通道数据实现融合,结合了两种模态的特性信息,最后对深度特征进行可视化,使得分类模型具有可解释性。实验证明,与单一模态的数据和临床通用方法相比,该算法在受试者工作特征曲线下的面积、准确率等指标上有更好的结果,模型性能更加稳定。该算法可以有效实现无创评估患者临床显著肝脏纤维化,具有广泛应用的价值。

Liver fibrosis is the main cause of liver diseases.Timely prediction of liver fibrosis is of great significance for the diagnosis and treatment of liver diseases.In view of the low diagnostic accuracy of serum biochemical indicators in traditional clinical examinations and the inability of visual evaluation magnetic resonance images to accurately grade the degree of liver fibrosis,a deep learning algorithm based on multi-modality and multi-channel fusion for liver fibrosis grading(MCMD-Resnet18)is designed.The algorithm is based on the idea of multi-channel and makes full use of the semantic information between magnetic resonance image data and clinical text data.The image channel is trained based on Resnet18 to obtain a feature map with rich semantic information.After global pooling,the data of image and text from different channel are fused in the classifier stage.The fusion module combines the characteristic information between the two modality.Then the depth features are visualized to make the classification model interpretable.Experiments show that compared with single-modality data and traditional clinical methods,the algorithm has better results in the area under the working characteristic curve of the subject,accuracy and other indicators,and the model performance is more stable.This algorithm can effectively achieve non-invasive assessment of clinically significant liver fibrosis in patients,and has wide application value.