A deep learning fusion model for accurate classification of brain tumours in Magnetic Resonance images

Detecting brain tumours is complex due to the natural variation in their location, shape, and intensity in images. While having accurate detection and segmentation of brain tumours would be beneficial, current methods still need to solve this problem despite the numerous available approaches. Precise analysis of Magnetic Resonance Imaging (MRI) is crucial for detecting, segmenting, and classifying brain tumours in medical diagnostics. Magnetic Resonance Imaging is a vital component in medical diagnosis, and it requires precise, efficient, careful, efficient, and reliable image analysis techniques. The authors developed a Deep Learning (DL) fusion model to classify brain tumours reliably. Deep Learning models require large amounts of training data to achieve good results, so the researchers utilised data augmentation techniques to increase the dataset size for training models. VGG16, ResNet50, and convolutional deep belief networks networks extracted deep features from MRI images. Softmax was used as the classifier, and the training set was supplemented with intentionally created MRI images of brain tumours in addition to the genuine ones. The features of two DL models were combined in the proposed model to generate a fusion model, which significantly increased classification accuracy. An openly accessible dataset from the internet was used to test the model's performance, and the experimental results showed that the proposed fusion model achieved a classification accuracy of 98.98%. Finally, the results were compared with existing methods, and the proposed model outperformed them significantly.

Brain Tumor Classification Using Image Fusion and EFPA-SVM Classifier

An accurate and early diagnosis of brain tumors based on medical ima-ging modalities is of great interest because brain tumors are a harmful threat to a person’s health worldwide.Several medical imaging techniques have been used to analyze brain tumors,including computed tomography(CT)and magnetic reso-nance imaging(MRI).CT provides information about dense tissues,whereas MRI gives information about soft tissues.However,the fusion of CT and MRI images has little effect on enhancing the accuracy of the diagnosis of brain tumors.Therefore,machine learning methods have been adopted to diagnose brain tumors in recent years.This paper intends to develop a novel scheme to detect and classify brain tumors based on fused CT and MRI images.The pro-posed approach starts with preprocessing the images to reduce the noise.Then,fusion rules are applied to get the fused image,and a segmentation algorithm is employed to isolate the tumor region from the background to isolate the tumor region.Finally,a machine learning classifier classified the brain images into benign and malignant tumors.Computing statistical measures evaluate the classi-fication potential of the proposed scheme.Experimental outcomes are provided,and the Enhanced Flower Pollination Algorithm(EFPA)system shows that it out-performs other brain tumor classification methods considered for comparison.

Consistent and Specific Multi-View Functional Brain Networks Fusion for Autism Spectrum Disorder Diagnosis

Functional brain networks (FBN) based on resting-state functional magnetic resonance imaging (rs-fMRI) have become an important tool for exploring underlying organization patterns in the brain, which can provide an objective basis for brain disorders such as autistic spectrum disorder (ASD). Due to its importance, researchers have proposed a number of FBN estimation methods. However, most existing methods only model a type of functional connection relationship between brain regions-of-interest (ROIs), such as partial correlation or full correlation, which is difficult to fully capture the subtle connections among ROIs since these connections are extremely complex. Motivated by the multi-view learning, in this study we propose a novel Consistent and Specific Multi-view FBNs Fusion (CSMF) approach. Concretely, we first construct multi-view FBNs (i.e., multiple types of FBNs modelling various relationships among ROIs), and then these FBNs are decomposed into a consistent representation matrix and their own specific matrices which capture their common and unique information, respectively. Lastly, to obtain a better brain representation, it is fusing the consistent and specific representation matrices in the latent representation spaces of FBNs, but not directly fusing the original FBNs. This potentially makes it more easily to find the comprehensively brain connections. The experimental results of ASD identification on the ABIDE datasets validate the effectiveness of our proposed method compared to several state-of-the-art methods. Our proposed CSMF method achieved 72.8% and 76.67% classification performance on the ABIDE dataset.

The role of snapin in regulation of brain homeostasis

Brain homeostasis refe rs to the normal working state of the brain in a certain period,which is impo rtant for overall health and normal life activities.Currently,there is a lack of effective treatment methods for the adverse consequences caused by brain homeostasis imbalance.Snapin is a protein that assists in the formation of neuronal synapses and plays a crucial role in the normal growth and development of synapses.Recently,many researchers have reported the association between snapin and neurologic and psychiatric disorders,demonstrating that snapin can improve brain homeostasis.Clinical manifestations of brain disease often involve imbalances in brain homeostasis and may lead to neurological and behavioral sequelae.This article aims to explo re the role of snapin in restoring brain homeostasis after injury or diseases,highlighting its significance in maintaining brain homeostasis and treating brain diseases.Additionally,it comprehensively discusses the implications of snapin in other extracerebral diseases such as diabetes and viral infections,with the objective of determining the clinical potential of snapin in maintaining brain homeostasis.

Brain Time Stack图像融合技术在CT中的应用

目的:分析Brain Time Stack图像融合技术在CT中的应用。方法:选取2021年3月—2022年9月衡水市第四人民医院收治的50例CT检查患者作为研究对象。所有患者进行CT检查并进行Brain Time Stack后处理。比较四组不同部位CT值、标准差(SD)、信噪比(SNR)。比较四组图像主观质量评分。分析不同部位CT值、SD、SNR与图像主观质量评分的相关性。结果:B组的延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显低于A组;C组的延髓、脑室、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值高于A组;D组延髓、额叶灰质、颞肌肌肉CT值明显低于A组,脑室、额叶白质、小脑外侧CT值明显高于A组;C组延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显高于B组;D组延髓、脑室、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显高于B组;D组延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显低于C组;D组脑室CT值明显高于C组,差异有统计学意义(P<0.05)。B组、C组、D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SD值明显低于A组;C组延髓、脑室、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SD值均明显高于B组;C组额叶灰质SD明显低于B组;D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、肌肉SD均明显低于B组、C组,差异有统计学意义(P<0.05)。B组、C组、D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SNR均明显高于A组;C组、D组延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SNR值明显高于B组;C组、D组脑室SNR明显低于B组;D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SNR明显高于C组,差异有统计学意义(P<0.05)。D组图像主观质量评分最高,差异有统计学意义(P<0.05)。延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧及颞肌肌肉SD与主观质量评分呈明显负相关,SNR与主观质量评分间呈明显正相关,差异有统计学意义(P<0.05)。结论:利用Brain Time Stack图像融合技术对头部CT扫描检查图像处理,动脉期结合前一期及后一期的图像数据在处理后具有更好的质量和更少的噪音。

Application of Preoperative CT/MRI Image Fusion in Target Positioning for Deep Brain Stimulation

Objective To explore the efficacy of target positioning by preoperative CT/MRI image fusion technique in deep brain stimulation.Methods We retrospectively analyzed the clinical data and images of 79 cases(68 with Parkinson's disease,11 with dystonia) who received preoperative CT/MRI image fusion in target positioning of subthalamic nucleus in deep brain stimulation.Deviation of implanted electrodes from the target nucleus of each patient were measured.Neurological evaluations of each patient before and after the treatment were performed and compared.Complications of the positioning and treatment were recorded.Results The mean deviations of the electrodes implanted on X,Y,and Z axis were 0.5 mm,0.6 mm,and 0.6 mm,respectively.Postoperative neurologic evaluations scores of unified Parkinson's disease rating scale(UPDRS) for Parkinson's disease and Burke-Fahn-Marsden Dystonia Rating Scale(BFMDRS) for dystonia patients improved significantly compared to the preoperative scores(P<0.001); Complications occurred in 10.1%(8/79) patients,and main side effects were dysarthria and diplopia.Conclusion Target positioning by preoperative CT/MRI image fusion technique in deep brain stimulation has high accuracy and good clinical outcomes.

Optimal Fusion-Based Handcrafted with Deep Features for Brain Cancer Classification

Brain cancer detection and classification is done utilizing distinct medical imaging modalities like computed tomography(CT),or magnetic resonance imaging(MRI).An automated brain cancer classification using computer aided diagnosis(CAD)models can be designed to assist radiologists.With the recent advancement in computer vision(CV)and deep learning(DL)models,it is possible to automatically detect the tumor from images using a computer-aided design.This study focuses on the design of automated Henry Gas Solubility Optimization with Fusion of Handcrafted and Deep Features(HGSO-FHDF)technique for brain cancer classification.The proposed HGSO-FHDF technique aims for detecting and classifying different stages of brain tumors.The proposed HGSO-FHDF technique involves Gabor filtering(GF)technique for removing the noise and enhancing the quality of MRI images.In addition,Tsallis entropy based image segmentation approach is applied to determine injured brain regions in the MRI image.Moreover,a fusion of handcrafted with deep features using Residual Network(ResNet)is utilized as feature extractors.Finally,HGSO algorithm with kernel extreme learning machine(KELM)model was utilized for identifying the presence of brain tumors.For examining the enhanced brain tumor classification performance,a comprehensive set of simulations take place on the BRATS 2015 dataset.

Deep Learning-Enhanced Brain Tumor Prediction via Entropy-Coded BPSO in CIELAB Color Space

Early detection of brain tumors is critical for effective treatment planning.Identifying tumors in their nascent stages can significantly enhance the chances of patient survival.While there are various types of brain tumors,each with unique characteristics and treatment protocols,tumors are often minuscule during their initial stages,making manual diagnosis challenging,time-consuming,and potentially ambiguous.Current techniques predominantly used in hospitals involve manual detection via MRI scans,which can be costly,error-prone,and time-intensive.An automated system for detecting brain tumors could be pivotal in identifying the disease in its earliest phases.This research applies several data augmentation techniques to enhance the dataset for diagnosis,including rotations of 90 and 180 degrees and inverting along vertical and horizontal axes.The CIELAB color space is employed for tumor image selection and ROI determination.Several deep learning models,such as DarkNet-53 and AlexNet,are applied to extract features from the fully connected layers,following the feature selection using entropy-coded Particle Swarm Optimization(PSO).The selected features are further processed through multiple SVM kernels for classification.This study furthers medical imaging with its automated approach to brain tumor detection,significantly minimizing the time and cost of a manual diagnosis.Our method heightens the possibilities of an earlier tumor identification,creating an avenue for more successful treatment planning and better overall patient outcomes.

多模态图像融合及三维重建技术在后颅窝肿瘤手术中的应用

目的旨在探讨多模态图像融合及三维重建技术在后颅窝肿瘤手术治疗中的应用效果。方法回顾性分析2022年1月至2023年9月在江苏大学附属人民医院神经外科接受手术治疗的19例后颅窝肿瘤患者的临床资料。所有患者在术前均接受头部CT和MRI检查,并将影像数据输入影像融合工作站进行图像融合和三维重建。医生利用这些融合后的影像进行肿瘤空间评估和模拟手术入路。术后统计肿瘤全切除率和术后并发症,并对其应用价值进行评估。结果多模态图像融合及三维重建技术能清晰显示后颅窝肿瘤与周围结构的解剖关系,19例患者中肿瘤全切除15例(78.9%),次全切除4例,无围手术期死亡患者。术后并发症包括脑水肿2例,颅内感染1例,面瘫2例,吞咽困难1例。根据医生的反馈,多模态图像融合及三维重建技术在手术中表现出显著价值的案例有16例,辅助价值的案例有3例。结论多模态图像融合及三维重建技术可精准、清晰地显示后颅窝肿瘤与周围重要组织的空间关系,有助于医生设计更精准的手术切口和选择更合理的手术入路,对手术顺利完成有较高的辅助价值。

融合CNN与Transformer的MRI脑肿瘤图像分割

为解决卷积神经网络(convolutional neural network,CNN)在学习全局上下文信息和边缘细节方面受到很大限制的问题,提出一种同时学习局语义信息和局部空间细节的级联神经网络用于脑肿瘤医学图像分割。首先将输入体素分别送入CNN和Transformer分支,在编码阶段结束后,采用一种双分支融合模块将2个分支学习到的特征有效地结合起来以实现全局信息与局部信息的融合。双分支融合模块利用哈达玛积对双分支特征之间的细粒度交互进行建模,同时使用多重注意力机制充分提取特征图通道和空间信息并抑制无效的噪声信息。在BraTS竞赛官网评估了本文方法,在BraTS2019验证集上增强型肿瘤区、全肿瘤区和肿瘤核心区的Dice分数分别为77.92%,89.20%和81.20%。相较于其他先进的三维医学图像分割方法,本文方法表现出了更好的分割性能,为临床医生做出准确的脑肿瘤细胞评估和治疗方案提供了可靠依据。

基于多模态融合和自适应剪枝Transformer的脑肿瘤图像分割算法

脑肿瘤是目前世界上最致命的肿瘤之一,所以脑肿瘤图像的自动分割在临床诊疗中变得日益重要.近年来,基于CNN和Transformer的脑肿瘤分割方法在医学图像分割领域取得了令人欣喜的成就.然而,大多数方法没有充分利用脑肿瘤多模态间的互补性和差异性,并且模型中的Transformer在捕获远程依赖性的同时,忽略了其较大的计算复杂性、冗余依赖性等问题.针对此问题,提出一种基于多模态融合和自适应剪枝Transformer的脑肿瘤图像分割方法(MF-MAPT Swin UNETR),其中多模态融合模块可以充分学习性质相近的模态间信息和不同模态不同尺度的特征变化,为后续分割提供了充分的准备;基于多模态的自适应剪枝Transformer可以降低计算复杂度,对提升性能有一定的帮助,将MF-MAPT Swin UNETR模型在两个公共数据集上进行了实验验证,结果表明,该模型较最先进的方法整体具有突出的分割性能.

跨模态融合的双注意力脑肿瘤分割算法

针对脑肿瘤多模态信息融合不充分以及肿瘤区域细节信息丢失等问题,提出了一种跨模态融合的双注意力脑肿瘤图像分割网络(CFDA-Net).在编码器-解码器的基础结构上,首先在编码器分支采用密集块与大内核注意力并行的新卷积块,可以使全局和局部信息有效融合且可以防止反向传播时梯度消失的问题;其次在编码器的第2、3和4层的左侧加入多模态深度融合模块,有效地利用不同模态间的互补信息;然后在解码器分支使用Shuffle Attention注意力将特征图分组处理后再聚合,其中分组的子特征一分为二地获取空间与通道的重要注意特征.最后使用二进制交叉熵(binary cross entropy,BCE)、Dice Loss与L2 Loss组成新的混合损失函数,缓解了脑肿瘤数据的类别不平衡问题,进一步提升分割性能.在BraTS2019脑肿瘤数据集上的实验结果表明,该模型在整体肿瘤区域、肿瘤核心区域和肿瘤增强区域的平均Dice系数值分别为0.887、0.892和0.815.与其他先进的分割方法ADHDC-Net、SDS-MSA-Net等相比,该模型在肿瘤核心区域和增强区域具有更好的分割效果.

基于多脑区注意力机制胶囊融合网络的EEG-fNIRS情感识别

为了提高情感识别的准确率,提出多脑区注意力机制和胶囊融合模块的胶囊网络模型(MBA-CFc CapsNet).通过情感视频片段诱发采集EEG-fNIRS信号,构建TYUT3.0数据集.提取EEG和f NIRS的特征,将其映射到矩阵,通过多脑区注意力机制融合EEG和fNIRS的特征,给予不同脑区特征不同的权重,以提取质量更高的初级胶囊.使用胶囊融合模块,减少进入动态路由机制的胶囊数量,减少模型运行的时间.利用MBA-CFc CapsNet模型在TYUT3.0情感数据集上进行实验,与单模态EEG和f NIRS识别结果相比,2种信号结合情感识别的准确率提高了1.53%和14.35%.MBA-CF-cCapsNet模型与原始CapsNet模型相比,平均识别率提高了4.98%,与当前常用的CapsNet情感识别模型相比提高了1%~5%.

脑机融合增强视觉表征:从“脑在环路建模”到“脑不在环路应用”

基于深度神经网络特征及恒河猴大脑视觉皮层脑电响应,提出了基于自适应信息融合方法的“脑在环路”图像表征模型,证明了脑机融合可以提供互补信息并提升深度神经网络模型性能;提出了基于大脑响应重建和共享表征空间的两种脑机融合计算模式,实现了“脑在环路建模,脑不在环路应用”,拓宽了脑机融合模型的应用场景;通过特征显著性可视化方法证明了共享表征有效性,为充分利用大脑视觉表征的神经响应提供了新思路。

采用多任务特征融合的脑电情绪识别方法

特征选择与融合是提升脑电信号情绪解码精度的重要手段之一。然而,当前脑电情绪解码中的特征选择方法常忽略了脑电信号内在数据结构的隐含信息。该文提出一种基于近邻传播聚类的多任务特征融合方法,通过L_(2,1)范数约束实现稀疏特征选择,同时利用图拉普拉斯正则化保持不同子类间的潜在关系。该算法在不揭示真实样本标签的情况下,在子任务空间有效融合脑网络空间拓扑结构信息和微分熵信息,为高精度脑电信号情绪解码提供具有更高情绪表征能力的特征。DEAP和SEED数据集以及本实验室数据集的分析结果表明,该文提出的方法能显著提高脑电情绪解码的精度。

城市智慧公交云脑建设与应用

针对公交海量数据多源、异构、分散,不能充分收集、治理和应用,传统智能公交不同系统间联动性不强的实际现状,本文通过对北京、杭州、广州、上海等城市数据大脑、云脑平台的调查研究,结合云计算、人工智能等新一代信息技术,提出智慧公交云脑平台建设的总体架构,深挖十大核心应用功能,分析六大发展趋势。通过多源数据的融合、挖掘,核心模型算法的搭建,支撑公交业务应用,并加强各应用系统间的信息耦合。

基于有效感受野和注意力融合机制的脑肿瘤全自动分割

深度学习在医学图像分割领域取得了显著成果,但其在脑肿瘤分割任务中,仍面临感受野不足、冗余信息过多、信息丢失等问题;为此,本研究提出一种基于编-解码结构的脑肿瘤分割网络模型(EAU-Net)。EAU-Net采用有效感受野拓展模块和注意力融合模块改善脑肿瘤分割网络感受野不足与冗余信息过多带来的不利影响;同时,引入基于倒残差结构的瓶颈重采样模块,有效避免上下采样时造成的信息损失,并采用深度卷积降低网络的计算量。在BraTS2020数据集上的实验结果表明,EAU-Net获得最优的分割精度,验证了其在脑肿瘤分割任务中的可行性和有效性。

基于多模态模糊特征融合的脑龄协同预测算法

深度神经网络可通过训练从大脑图像中预测年龄,作为识别衰老相关疾病的生物标志物.传统的脑龄预测方法往往依赖于单一模态的图像数据,而多模态数据可提供更全面的信息,提高预测精度.然而,现有的多模态融合方法往往不能充分利用不同模态之间的相关性和互补性.为了克服上述问题,文中提出基于多模态模糊特征融合的脑龄协同预测算法(CMFF),设计模糊融合模块和多模态协同卷积模块,可有效利用多模态信息之间的相关信息和互补信息.首先,利用卷积神经网络从多模态脑图中提取特征张量,径向拼接后整合到一个全局特征张量中.然后,利用模糊融合模块学习被模糊化的特征,再将特征应用到多模态协同卷积模块,通过特定的卷积层增强模态间的互补信息.最后,基于性别信息和经过模糊协同处理的特征执行年龄预测回归任务,得到准确的预测年龄.在SRPBS多重障碍MRI数据集上的实验表明,CMFF性能较优.

Multimodal Fusion of Brain Imaging Data: Methods and Applications

Neuroimaging data typically include multiple modalities,such as structural or functional magnetic resonance imaging,dif-fusion tensor imaging,and positron emission tomography,which provide multiple views for observing and analyzing the brain.To lever-age the complementary representations of different modalities,multimodal fusion is consequently needed to dig out both inter-modality and intra-modality information.With the exploited rich information,it is becoming popular to combine multiple modality data to ex-plore the structural and functional characteristics of the brain in both health and disease status.In this paper,we first review a wide spectrum of advanced machine learning methodologies for fusing multimodal brain imaging data,broadly categorized into unsupervised and supervised learning strategies.Followed by this,some representative applications are discussed,including how they help to under-stand the brain arealization,how they improve the prediction of behavioral phenotypes and brain aging,and how they accelerate the biomarker exploration of brain diseases.Finally,we discuss some exciting emerging trends and important future directions.Collectively,we intend to offer a comprehensive overview of brain imaging fusion methods and their successful applications,along with the chal-lenges imposed by multi-scale and big data,which arises an urgent demand on developing new models and platforms.

基于距匹配及判别表征学习的多模态特征融合分类模型研究:高级别胶质瘤与单发性脑转移瘤的鉴别诊断

目的探索基于距匹配及判别表征学习的多模态特征融合分类模型在鉴别高级别胶质瘤(HGG)与单发性脑转移(SBM)中的鉴别能力和应用价值。方法收集了121例患者(61例HGG和60例SBM)的多参数磁共振成像(MRI)扫描图像,在T1W1、T2W1、T2加权液体衰减反转恢复(T2_FLAIR)和T1WI增强图像(CE_T1WI)4种常规轴位MRI图像上勾画目标感兴趣区域(ROI),并使用开源影像组学工具Pyradiomics从4个MRI序列分别提取影像组学特征。使用本研究提出的基于距匹配及判别表征学习的多模态特征融合分类模型对4个MRI序列的影像组学特征进行融合并得到分类模型。采用五折交叉验证方法和特异性(SPE)、灵敏度(SEN)、准确率(ACC)、ROC曲线下面积(AUC)评价该分类模型的鉴别性能。将本研究所提模型与其他特征融合分类模型对于HGG与SBM的鉴别能力进行定量比较,同时对本研究提出特征融合方法得到的融合特征进行样本散点可视化实验,验证本研究所提出的多模态特征融合分类模型的可行性和有效性。结果五折交叉验证结果显示本研究所提出的基于距匹配及判别表征学习的多模态特征融合分类模型在鉴别高级别胶质瘤与单发性脑转移瘤中的SPE、SEN、ACC、AUC分别为:0.871、0.817、0.843、0.930,且特征融合方法在可视化实验中具有优秀的表现。结论基于距匹配及判别表征学习的多模态特征融合分类模型在鉴别高级别胶质瘤与单发性脑转移瘤中的应用具有优秀的鉴别能力和较高的应用价值。同时,与其他特征融合分类模型相比,本研究提出的分类模型在HGG与SBM的鉴别分类任务中具有较大的优势。