A deep learning fusion model for accurate classification of brain tumours in Magnetic Resonance images

Detecting brain tumours is complex due to the natural variation in their location, shape, and intensity in images. While having accurate detection and segmentation of brain tumours would be beneficial, current methods still need to solve this problem despite the numerous available approaches. Precise analysis of Magnetic Resonance Imaging (MRI) is crucial for detecting, segmenting, and classifying brain tumours in medical diagnostics. Magnetic Resonance Imaging is a vital component in medical diagnosis, and it requires precise, efficient, careful, efficient, and reliable image analysis techniques. The authors developed a Deep Learning (DL) fusion model to classify brain tumours reliably. Deep Learning models require large amounts of training data to achieve good results, so the researchers utilised data augmentation techniques to increase the dataset size for training models. VGG16, ResNet50, and convolutional deep belief networks networks extracted deep features from MRI images. Softmax was used as the classifier, and the training set was supplemented with intentionally created MRI images of brain tumours in addition to the genuine ones. The features of two DL models were combined in the proposed model to generate a fusion model, which significantly increased classification accuracy. An openly accessible dataset from the internet was used to test the model's performance, and the experimental results showed that the proposed fusion model achieved a classification accuracy of 98.98%. Finally, the results were compared with existing methods, and the proposed model outperformed them significantly.

Brain Tumor Classification Using Image Fusion and EFPA-SVM Classifier

An accurate and early diagnosis of brain tumors based on medical ima-ging modalities is of great interest because brain tumors are a harmful threat to a person’s health worldwide.Several medical imaging techniques have been used to analyze brain tumors,including computed tomography(CT)and magnetic reso-nance imaging(MRI).CT provides information about dense tissues,whereas MRI gives information about soft tissues.However,the fusion of CT and MRI images has little effect on enhancing the accuracy of the diagnosis of brain tumors.Therefore,machine learning methods have been adopted to diagnose brain tumors in recent years.This paper intends to develop a novel scheme to detect and classify brain tumors based on fused CT and MRI images.The pro-posed approach starts with preprocessing the images to reduce the noise.Then,fusion rules are applied to get the fused image,and a segmentation algorithm is employed to isolate the tumor region from the background to isolate the tumor region.Finally,a machine learning classifier classified the brain images into benign and malignant tumors.Computing statistical measures evaluate the classi-fication potential of the proposed scheme.Experimental outcomes are provided,and the Enhanced Flower Pollination Algorithm(EFPA)system shows that it out-performs other brain tumor classification methods considered for comparison.

Application of Preoperative CT/MRI Image Fusion in Target Positioning for Deep Brain Stimulation

Objective To explore the efficacy of target positioning by preoperative CT/MRI image fusion technique in deep brain stimulation.Methods We retrospectively analyzed the clinical data and images of 79 cases(68 with Parkinson's disease,11 with dystonia) who received preoperative CT/MRI image fusion in target positioning of subthalamic nucleus in deep brain stimulation.Deviation of implanted electrodes from the target nucleus of each patient were measured.Neurological evaluations of each patient before and after the treatment were performed and compared.Complications of the positioning and treatment were recorded.Results The mean deviations of the electrodes implanted on X,Y,and Z axis were 0.5 mm,0.6 mm,and 0.6 mm,respectively.Postoperative neurologic evaluations scores of unified Parkinson's disease rating scale(UPDRS) for Parkinson's disease and Burke-Fahn-Marsden Dystonia Rating Scale(BFMDRS) for dystonia patients improved significantly compared to the preoperative scores(P<0.001); Complications occurred in 10.1%(8/79) patients,and main side effects were dysarthria and diplopia.Conclusion Target positioning by preoperative CT/MRI image fusion technique in deep brain stimulation has high accuracy and good clinical outcomes.

Brain Time Stack图像融合技术在CT中的应用

目的:分析Brain Time Stack图像融合技术在CT中的应用。方法:选取2021年3月—2022年9月衡水市第四人民医院收治的50例CT检查患者作为研究对象。所有患者进行CT检查并进行Brain Time Stack后处理。比较四组不同部位CT值、标准差(SD)、信噪比(SNR)。比较四组图像主观质量评分。分析不同部位CT值、SD、SNR与图像主观质量评分的相关性。结果:B组的延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显低于A组;C组的延髓、脑室、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值高于A组;D组延髓、额叶灰质、颞肌肌肉CT值明显低于A组,脑室、额叶白质、小脑外侧CT值明显高于A组;C组延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显高于B组;D组延髓、脑室、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显高于B组;D组延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉CT值明显低于C组;D组脑室CT值明显高于C组,差异有统计学意义(P<0.05)。B组、C组、D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SD值明显低于A组;C组延髓、脑室、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SD值均明显高于B组;C组额叶灰质SD明显低于B组;D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、肌肉SD均明显低于B组、C组,差异有统计学意义(P<0.05)。B组、C组、D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SNR均明显高于A组;C组、D组延髓、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SNR值明显高于B组;C组、D组脑室SNR明显低于B组;D组延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧、颞肌肌肉SNR明显高于C组,差异有统计学意义(P<0.05)。D组图像主观质量评分最高,差异有统计学意义(P<0.05)。延髓、脑室、额叶灰质、额叶白质、小脑内侧、小脑外侧及颞肌肌肉SD与主观质量评分呈明显负相关,SNR与主观质量评分间呈明显正相关,差异有统计学意义(P<0.05)。结论:利用Brain Time Stack图像融合技术对头部CT扫描检查图像处理,动脉期结合前一期及后一期的图像数据在处理后具有更好的质量和更少的噪音。

Test method of laser paint removal based on multi-modal feature fusion

Laser cleaning is a highly nonlinear physical process for solving poor single-modal(e.g., acoustic or vision)detection performance and low inter-information utilization. In this study, a multi-modal feature fusion network model was constructed based on a laser paint removal experiment. The alignment of heterogeneous data under different modals was solved by combining the piecewise aggregate approximation and gramian angular field. Moreover, the attention mechanism was introduced to optimize the dual-path network and dense connection network, enabling the sampling characteristics to be extracted and integrated. Consequently, the multi-modal discriminant detection of laser paint removal was realized. According to the experimental results, the verification accuracy of the constructed model on the experimental dataset was 99.17%, which is 5.77% higher than the optimal single-modal detection results of the laser paint removal. The feature extraction network was optimized by the attention mechanism, and the model accuracy was increased by 3.3%. Results verify the improved classification performance of the constructed multi-modal feature fusion model in detecting laser paint removal, the effective integration of acoustic data and visual image data, and the accurate detection of laser paint removal.

Optimal Fusion-Based Handcrafted with Deep Features for Brain Cancer Classification

Brain cancer detection and classification is done utilizing distinct medical imaging modalities like computed tomography(CT),or magnetic resonance imaging(MRI).An automated brain cancer classification using computer aided diagnosis(CAD)models can be designed to assist radiologists.With the recent advancement in computer vision(CV)and deep learning(DL)models,it is possible to automatically detect the tumor from images using a computer-aided design.This study focuses on the design of automated Henry Gas Solubility Optimization with Fusion of Handcrafted and Deep Features(HGSO-FHDF)technique for brain cancer classification.The proposed HGSO-FHDF technique aims for detecting and classifying different stages of brain tumors.The proposed HGSO-FHDF technique involves Gabor filtering(GF)technique for removing the noise and enhancing the quality of MRI images.In addition,Tsallis entropy based image segmentation approach is applied to determine injured brain regions in the MRI image.Moreover,a fusion of handcrafted with deep features using Residual Network(ResNet)is utilized as feature extractors.Finally,HGSO algorithm with kernel extreme learning machine(KELM)model was utilized for identifying the presence of brain tumors.For examining the enhanced brain tumor classification performance,a comprehensive set of simulations take place on the BRATS 2015 dataset.

A Hybrid De-Noising Method on LASCA Images of Blood Vessels

A de-noising approach is proposed that based on the combination of wiener filtering, nonlinear filtering and wavelet fusion, which de-noise the LASCA (LAser Speckle Contrast Analysis) image of blood vessels in Small Animals. The approach first performs laser spectral contrast analysis on brain blood flow in rats, get their spatial and temporal contrast images. Then, a de-noising filtering method is proposed to deal with noise in LASCA. The image restoration is achieved by applying the proposed admixture filtering, and the subjective estimation and objective estimation are given to the de-noising images. As our experimental results shown, the proposed method provides clearer subjective sense and improved to over 25 db for PSNR.

多模态图像融合及三维重建技术在后颅窝肿瘤手术中的应用

目的旨在探讨多模态图像融合及三维重建技术在后颅窝肿瘤手术治疗中的应用效果。方法回顾性分析2022年1月至2023年9月在江苏大学附属人民医院神经外科接受手术治疗的19例后颅窝肿瘤患者的临床资料。所有患者在术前均接受头部CT和MRI检查,并将影像数据输入影像融合工作站进行图像融合和三维重建。医生利用这些融合后的影像进行肿瘤空间评估和模拟手术入路。术后统计肿瘤全切除率和术后并发症,并对其应用价值进行评估。结果多模态图像融合及三维重建技术能清晰显示后颅窝肿瘤与周围结构的解剖关系,19例患者中肿瘤全切除15例(78.9%),次全切除4例,无围手术期死亡患者。术后并发症包括脑水肿2例,颅内感染1例,面瘫2例,吞咽困难1例。根据医生的反馈,多模态图像融合及三维重建技术在手术中表现出显著价值的案例有16例,辅助价值的案例有3例。结论多模态图像融合及三维重建技术可精准、清晰地显示后颅窝肿瘤与周围重要组织的空间关系,有助于医生设计更精准的手术切口和选择更合理的手术入路,对手术顺利完成有较高的辅助价值。

融合CNN与Transformer的MRI脑肿瘤图像分割

为解决卷积神经网络(convolutional neural network,CNN)在学习全局上下文信息和边缘细节方面受到很大限制的问题,提出一种同时学习局语义信息和局部空间细节的级联神经网络用于脑肿瘤医学图像分割。首先将输入体素分别送入CNN和Transformer分支,在编码阶段结束后,采用一种双分支融合模块将2个分支学习到的特征有效地结合起来以实现全局信息与局部信息的融合。双分支融合模块利用哈达玛积对双分支特征之间的细粒度交互进行建模,同时使用多重注意力机制充分提取特征图通道和空间信息并抑制无效的噪声信息。在BraTS竞赛官网评估了本文方法,在BraTS2019验证集上增强型肿瘤区、全肿瘤区和肿瘤核心区的Dice分数分别为77.92%,89.20%和81.20%。相较于其他先进的三维医学图像分割方法,本文方法表现出了更好的分割性能,为临床医生做出准确的脑肿瘤细胞评估和治疗方案提供了可靠依据。

Image De-occlusion via Event-enhanced Multi-modal Fusion Hybrid Network

Seeing through dense occlusions and reconstructing scene images is an important but challenging task.Traditional framebased image de-occlusion methods may lead to fatal errors when facing extremely dense occlusions due to the lack of valid information available from the limited input occluded frames.Event cameras are bio-inspired vision sensors that record the brightness changes at each pixel asynchronously with high temporal resolution.However,synthesizing images solely from event streams is ill-posed since only the brightness changes are recorded in the event stream,and the initial brightness is unknown.In this paper,we propose an event-enhanced multi-modal fusion hybrid network for image de-occlusion,which uses event streams to provide complete scene information and frames to provide color and texture information.An event stream encoder based on the spiking neural network(SNN)is proposed to encode and denoise the event stream efficiently.A comparison loss is proposed to generate clearer results.Experimental results on a largescale event-based and frame-based image de-occlusion dataset demonstrate that our proposed method achieves state-of-the-art performance.

跨模态融合的双注意力脑肿瘤分割算法

针对脑肿瘤多模态信息融合不充分以及肿瘤区域细节信息丢失等问题,提出了一种跨模态融合的双注意力脑肿瘤图像分割网络(CFDA-Net).在编码器-解码器的基础结构上,首先在编码器分支采用密集块与大内核注意力并行的新卷积块,可以使全局和局部信息有效融合且可以防止反向传播时梯度消失的问题;其次在编码器的第2、3和4层的左侧加入多模态深度融合模块,有效地利用不同模态间的互补信息;然后在解码器分支使用Shuffle Attention注意力将特征图分组处理后再聚合,其中分组的子特征一分为二地获取空间与通道的重要注意特征.最后使用二进制交叉熵(binary cross entropy,BCE)、Dice Loss与L2 Loss组成新的混合损失函数,缓解了脑肿瘤数据的类别不平衡问题,进一步提升分割性能.在BraTS2019脑肿瘤数据集上的实验结果表明,该模型在整体肿瘤区域、肿瘤核心区域和肿瘤增强区域的平均Dice系数值分别为0.887、0.892和0.815.与其他先进的分割方法ADHDC-Net、SDS-MSA-Net等相比,该模型在肿瘤核心区域和增强区域具有更好的分割效果.

基于多模态模糊特征融合的脑龄协同预测算法

深度神经网络可通过训练从大脑图像中预测年龄,作为识别衰老相关疾病的生物标志物.传统的脑龄预测方法往往依赖于单一模态的图像数据,而多模态数据可提供更全面的信息,提高预测精度.然而,现有的多模态融合方法往往不能充分利用不同模态之间的相关性和互补性.为了克服上述问题,文中提出基于多模态模糊特征融合的脑龄协同预测算法(CMFF),设计模糊融合模块和多模态协同卷积模块,可有效利用多模态信息之间的相关信息和互补信息.首先,利用卷积神经网络从多模态脑图中提取特征张量,径向拼接后整合到一个全局特征张量中.然后,利用模糊融合模块学习被模糊化的特征,再将特征应用到多模态协同卷积模块,通过特定的卷积层增强模态间的互补信息.最后,基于性别信息和经过模糊协同处理的特征执行年龄预测回归任务,得到准确的预测年龄.在SRPBS多重障碍MRI数据集上的实验表明,CMFF性能较优.

结合扩张金字塔的脑部医学图像融合

针对现有脑部医学图像融合算法存在的融合图像细节模糊和边缘性差等问题,设计一种扩张金字塔特征提取算法,由特征提取器、特征融合器和特征重构器3部分组成。特征提取器由扩张金字塔特征模块提取浅层和深层图像特征的结合,防止图像细节信息的丢失;特征融合器采用改进的功能能量比(Functional Energy Ratio,FER)特征融合策略增强融合图像边缘信息;特征重构器由4层卷积构成归一化图像。实验结果表明,相较于当前通用的脑部融合算法,所提出的算法具有较好的视觉效果和细节信息,客观评价指标有更好的表现。

基于残差孪生网络的多模态脑肿瘤三维分割算法

为充分利用多模态医学影像间的关联性和互补性,精准分割脑肿瘤区域及评估预后效果,本研究提出一种基于残差孪生网络的多模态脑肿瘤三维分割模型。首先,利用残差孪生编码挖掘不同模态数据间的关联细节语义信息,并在编码路径间加入级联结构,优化层次间信息交互方式;其次,提出了多尺度像素注意力融合模块,以获取不同感受野的加权融合特征,并促进多个模态间的互补信息交流;最后,在解码阶段设计基于残差孪生编码结构的跳跃连接和注意力单元,引导模型关注与肿瘤分割相关的信息,进一步提升模型的分割性能。本研究在BraTS 2021数据集上进行了验证,在整体肿瘤、肿瘤核心和增强肿瘤三个区域的平均Dice系数分别达到0.928、0.914和0.879。本研究有望为临床脑疾病的早期诊断提供一种新的方法和思路。

Explainable Conformer Network for Detection of COVID-19 Pneumonia from Chest CT Scan: From Concepts toward Clinical Explainability

The early implementation of treatment therapies necessitates the swift and precise identification of COVID-19 pneumonia by the analysis of chest CT scans.This study aims to investigate the indispensable need for precise and interpretable diagnostic tools for improving clinical decision-making for COVID-19 diagnosis.This paper proposes a novel deep learning approach,called Conformer Network,for explainable discrimination of viral pneumonia depending on the lung Region of Infections(ROI)within a single modality radiographic CT scan.Firstly,an efficient U-shaped transformer network is integrated for lung image segmentation.Then,a robust transfer learning technique is introduced to design a robust feature extractor based on pre-trained lightweight Big Transfer(BiT-L)and finetuned on medical data to effectively learn the patterns of infection in the input image.Secondly,this work presents a visual explanation method to guarantee clinical explainability for decisions made by Conformer Network.Experimental evaluation of real-world CT data demonstrated that the diagnostic accuracy of ourmodel outperforms cutting-edge studies with statistical significance.The Conformer Network achieves 97.40% of detection accuracy under cross-validation settings.Our model not only achieves high sensitivity and specificity but also affords visualizations of salient features contributing to each classification decision,enhancing the overall transparency and trustworthiness of our model.The findings provide obvious implications for the ability of our model to empower clinical staff by generating transparent intuitions about the features driving diagnostic decisions.

基于改进TransUNet模型的脑肿瘤图像分割方法研究

针对肿瘤细胞图像与正常组织图像之间具有强相似性、边界模糊以及染色变化大等特点,提出了基于TransUNet网络的优化改进分割模型。此分割模型在以TransUNet为主干网络的基础上于编码器部分引入注意力机制,抑制不相关的部分以突显深层特征的语义信息。同时,改变上采样过程中的融合方式,引入BiFusion模块进行选择性地融合,从而使特征数据能够保留更多高分辨率细节信息。该分割模型在Kaggle脑部低级别胶质瘤数据集上验证。实验结果表明,改进后算法的均交并比,召回率和平均精度均值分别为:97.31%,99.91%和98.72%,与目前医学图像分割的主流方法相比具有更优的性能。

基于多序列MR图像融合的脑肿瘤自动分割算法

准确、高效的脑肿瘤分割,对脑肿瘤的精准诊断具有重要意义。由于脑肿瘤MR图像存在对比度低、易出现噪声、偏移场和容积效应等问题,现有脑肿瘤分割模型的分割精度较低。为了提高脑肿瘤分割精度,提出了基于双通道全卷积神经网络和条件随机场的多序列MR图像融合的脑肿瘤分割算法。双通道全卷积神经网络可提取更丰富的图像特征,条件随机场能克服训练过程的局部极小值和输入图片中噪声产生的不利影响。该算法在脑肿瘤分割挑战数据集BRATS2018中测试,其DSC、PPV、Sensitivity系数均较传统分割方法有显著提高。

Intelligent Breast Cancer Prediction Empowered with Fusion and Deep Learning

Breast cancer is the most frequently detected tumor that eventually could result in a significant increase in female mortality globally.According to clinical statistics,one woman out of eight is under the threat of breast cancer.Lifestyle and inheritance patterns may be a reason behind its spread among women.However,some preventive measures,such as tests and periodic clinical checks can mitigate its risk thereby,improving its survival chances substantially.Early diagnosis and initial stage treatment can help increase the survival rate.For that purpose,pathologists can gather support from nondestructive and efficient computer-aided diagnosis(CAD)systems.This study explores the breast cancer CAD method relying on multimodal medical imaging and decision-based fusion.In multimodal medical imaging fusion,a deep learning approach is applied,obtaining 97.5%accuracy with a 2.5%miss rate for breast cancer prediction.A deep extreme learning machine technique applied on feature-based data provided a 97.41%accuracy.Finally,decisionbased fusion applied to both breast cancer prediction models to diagnose its stages,resulted in an overall accuracy of 97.97%.The proposed system model provides more accurate results compared with other state-of-the-art approaches,rapidly diagnosing breast cancer to decrease its mortality rate.

VISUALIZATION OF HEAD AND NECK CANCER MODELS WITH A TRIPLE FUSION REPORTER GENE

The development of experimental animal models for head and neck tumors generally rely on the biol uminescence imaging to achieve the dynamic monitoring of the tumor growth and metastasis due to the complicated anatomical structures.Since the bioluminescence imaging is largely affected by the intracellular luciferase expression level and external D-luciferin concentrations,its imaging accuracy requires further confirmation.Here,a new triple fusion reportelr gene,which consists of a herpes simplex virus type 1 thymidine kinase(TK)gene for radioactive imaging,a far-red fuorescent protein(mLumin)gene for fuorescent imaging,and a firefly luciferase gene for bioluminescence imaging,was introduced for in vrivo observation of the head and neck tumors through multi-modality imaging.Results show that fuorescence and bioluminescence signals from mLumin and luciferase,respectively,were clearly observed in tumor cells,and TK could activate suicide pathway of the cells in the presence of nucleotide analog-ganciclovir(GCV),demonstrating the effecti veness of individual functions of each gene.Moreover,subcutaneous and metastasis animal models for head and neck tumors using the fusion reporter gene-expressing cell lines were established,allowing multi-modality imaging in vio.Together,the established tumor models of head and neck cancer based on the newly developed triple fusion reporter gene are ideal for monitoring tumor growth,assessing the drug therapeutic efficacy and verifying the effec-tiveness of new treatments.

融合多尺度特征与注意力的脑白质病变分割

针对目前磁共振脑影像上的脑白质病变分割精度较低、小病灶易漏识的问题,提出一种结合多尺度信息与注意力机制的U-Net改进模型用于脑白质病变分割.首先,引入多尺度卷积模块以拓展网络宽度,提升特征捕获能力.其次,引入混合下采样模块,对粗、细两种粒度的下采样特征进行融合以减少下采样过程中的信息损失;同时,引入跨层融合模块,通过对跳跃连接两端的编、解码信息进行融合,降低对等层间的语义差异.最后,在编码阶段采用分散注意力模式,根据深、浅层的不同特点分别设计空间注意力模块和通道注意力模块,以增强网络对病灶区域的关注度.在MICCAI2017 WMHs分割挑战赛提供的公开数据集上与同任务的其它文献算法进行对比,本文算法在召回率和相似系数的性能评估上均获得了有效提升,分别达到了0.834和0.803,这表明本文算法是一种有效的脑白质病变自动分割算法.