基于图学习的缺失脑网络生成及多模态融合诊断方法

Graph Learning-Based Methods for Generating Missing Brain Networks and Multi-modal Fusion Diagnosis

融合大脑结构和功能网络的多模态脑网络能够挖掘不同模态间的互补信息,有效提高癫痫等神经系统疾病的诊断准确率,在神经疾病诊断上具有优势。然而,由于多模态数据采集时间长、成本高,在实际应用中常面临模态缺失问题,导致可用数据量减少,模型的诊断精度和泛化能力下降。针对某一模态数据完全缺失问题,提出了基于图学习与循环一致生成对抗网络(Cycle-consistent generative adversarial networks,CycleGAN)的图CycleGAN方法。该方法通过引入图卷积神经网络与图注意力机制等图学习方法捕捉脑网络不同脑区间的特征信息,强化生成框架对图形式脑网络的特征提取能力,实现脑结构网络与功能网络的相互生成。此外,针对目前较少利用诊断结果评估生成数据质量的情况,提出了一种融合真实脑网络与生成脑网络的多模态融合分类模型,以进一步评估生成脑网络的有效性。在癫痫数据集上的实验结果表明,图CycleGAN方法能够有效利用已有的模态信息,实现缺失脑网络的生成。

The multi-modal brain network,which integrates the brain structural and functional networks,can effectively extract the complementary information from different modalities,significantly improving the diagnostic accuracy of neurological diseases such as epilepsy.However,due to the long acquisition time and high acquisition cost of multi-modal data collection,it often faces the problem of modality missingness in practical applications,leading to decreased diagnostic accuracy and generalization ability of the model.To address the issue of modality data completely missing,we propose a method based on graph learning methods and cycle-consistent generative adversarial networks,named Graph-CycleGAN method.This method captures feature information between different brain regions in the brain network by introducing graph neural networks,such as graph convolutional neural networks and graph attention mechanisms.Besides,it strengthens the feature extraction ability of the generative framework and realizes the mutual generation of brain structural network and functional network.In addition,to address the lack of diagnostic result-based evaluations for the quality of generated data,this paper proposes a classification model that integrates real and generated brain networks.Experimental results on the epilepsy dataset indicate that the proposed Graph-CycleGAN method can effectively realize the generation of missing brain network by utilizing the existing modality information.