一种基于多模态特征融合的骨质疏松评估方法

Osteoporosis Evaluation Method Based on Multimodal Feature Fusion

针对现有骨质疏松评估中诊断依据单一、准确率低的问题,综合考虑骨骼图像数据和问卷数据,首先提出一种基于深度神经网络的多模态特征融合骨质疏松评估方法;然后,针对骨骼图像特征较浅、结构固定的特点,使用Unet进行图像分割预处理,去除冗余信息以提升分类准确性;最后,针对普通卷积操作在把握全局信息方面的不足,提出采用基于non-local模块的卷积神经网络来进一步丰富特征信息.交叉验证结果表明,提出的多模态特征融合方法与仅单独使用图像数据或问卷数据的机器学习方法相比具有明显的优势,分类准确率分别提升了3.2%和22.3%.

Aiming at the problems that the problems of single diagnosis and low accuracy in the existing osteoporosis assessment,considering the bone image data and questionnaire data,a multi-modal feature fusion osteoporosis evaluation method based on deep neural network was proposed.And,for the characteristics of shallow image and fixed structure of bone image,Unet is used to perform image segmentation preprocessing to remove redundant information.In view of the shortcomings of ordinary convolution operations in grasping the global information,a new convolutional neural network based on non-local module was proposed to further enrich the feature information.Cross-validation shows that the proposed multimodal feature fusion method has obvious advantages compared with the machine learning method using only image data or questionnaire data alone,and the classification accuracy rate is increased by 3.2%and22.3%.