DCFNet:An Effective Dual-Branch Cross-Attention Fusion Network for Medical Image Segmentation

Automatic segmentation of medical images provides a reliable scientific basis for disease diagnosis and analysis.Notably,most existing methods that combine the strengths of convolutional neural networks(CNNs)and Transformers have made significant progress.However,there are some limitations in the current integration of CNN and Transformer technology in two key aspects.Firstly,most methods either overlook or fail to fully incorporate the complementary nature between local and global features.Secondly,the significance of integrating the multiscale encoder features from the dual-branch network to enhance the decoding features is often disregarded in methods that combine CNN and Transformer.To address this issue,we present a groundbreaking dual-branch cross-attention fusion network(DCFNet),which efficiently combines the power of Swin Transformer and CNN to generate complementary global and local features.We then designed the Feature Cross-Fusion(FCF)module to efficiently fuse local and global features.In the FCF,the utilization of the Channel-wise Cross-fusion Transformer(CCT)serves the purpose of aggregatingmulti-scale features,and the Feature FusionModule(FFM)is employed to effectively aggregate dual-branch prominent feature regions from the spatial perspective.Furthermore,within the decoding phase of the dual-branch network,our proposed Channel Attention Block(CAB)aims to emphasize the significance of the channel features between the up-sampled features and the features generated by the FCFmodule to enhance the details of the decoding.Experimental results demonstrate that DCFNet exhibits enhanced accuracy in segmentation performance.Compared to other state-of-the-art(SOTA)methods,our segmentation framework exhibits a superior level of competitiveness.DCFNet’s accurate segmentation of medical images can greatly assist medical professionals in making crucial diagnoses of lesion areas in advance.

A review of medical ocular image segmentation

Deep learning has been extensively applied to medical image segmentation,resulting in significant advancements in the field of deep neural networks for medical image segmentation since the notable success of U Net in 2015.However,the application of deep learning models to ocular medical image segmentation poses unique challenges,especially compared to other body parts,due to the complexity,small size,and blurriness of such images,coupled with the scarcity of data.This article aims to provide a comprehensive review of medical image segmentation from two perspectives:the development of deep network structures and the application of segmentation in ocular imaging.Initially,the article introduces an overview of medical imaging,data processing,and performance evaluation metrics.Subsequently,it analyzes recent developments in U-Net-based network structures.Finally,for the segmentation of ocular medical images,the application of deep learning is reviewed and categorized by the type of ocular tissue.

ATFF: Advanced Transformer with Multiscale Contextual Fusion for Medical Image Segmentation

Deep convolutional neural network (CNN) greatly promotes the automatic segmentation of medical images. However, due to the inherent properties of convolution operations, CNN usually cannot establish long-distance interdependence, which limits the segmentation performance. Transformer has been successfully applied to various computer vision, using self-attention mechanism to simulate long-distance interaction, so as to capture global information. However, self-attention lacks spatial location and high-performance computing. In order to solve the above problems, we develop a new medical transformer, which has a multi-scale context fusion function and can be used for medical image segmentation. The proposed model combines convolution operation and attention mechanism to form a u-shaped framework, which can capture both local and global information. First, the traditional converter module is improved to an advanced converter module, which uses post-layer normalization to obtain mild activation values, and uses scaled cosine attention with a moving window to obtain accurate spatial information. Secondly, we also introduce a deep supervision strategy to guide the model to fuse multi-scale feature information. It further enables the proposed model to effectively propagate feature information across layers, Thanks to this, it can achieve better segmentation performance while being more robust and efficient. The proposed model is evaluated on multiple medical image segmentation datasets. Experimental results demonstrate that the proposed model achieves better performance on a challenging dataset (ETIS) compared to existing methods that rely only on convolutional neural networks, transformers, or a combination of both. The mDice and mIou indicators increased by 2.74% and 3.3% respectively.

Application of U-Net and Optimized Clustering in Medical Image Segmentation:A Review

As a mainstream research direction in the field of image segmentation,medical image segmentation plays a key role in the quantification of lesions,three-dimensional reconstruction,region of interest extraction and so on.Compared with natural images,medical images have a variety of modes.Besides,the emphasis of information which is conveyed by images of different modes is quite different.Because it is time-consuming and inefficient to manually segment medical images only by professional and experienced doctors.Therefore,large quantities of automated medical image segmentation methods have been developed.However,until now,researchers have not developed a universal method for all types of medical image segmentation.This paper reviews the literature on segmentation techniques that have produced major breakthroughs in recent years.Among the large quantities of medical image segmentation methods,this paper mainly discusses two categories of medical image segmentation methods.One is the improved strategies based on traditional clustering method.The other is the research progress of the improved image segmentation network structure model based on U-Net.The power of technology proves that the performance of the deep learning-based method is significantly better than that of the traditional method.This paper discussed both advantages and disadvantages of different algorithms and detailed how these methods can be used for the segmentation of lesions or other organs and tissues,as well as possible technical trends for future work.

TC-Fuse: A Transformers Fusing CNNs Network for Medical Image Segmentation

In medical image segmentation task,convolutional neural networks(CNNs)are difficult to capture long-range dependencies,but transformers can model the long-range dependencies effectively.However,transformers have a flexible structure and seldom assume the structural bias of input data,so it is difficult for transformers to learn positional encoding of the medical images when using fewer images for training.To solve these problems,a dual branch structure is proposed.In one branch,Mix-Feed-Forward Network(Mix-FFN)and axial attention are adopted to capture long-range dependencies and keep the translation invariance of the model.Mix-FFN whose depth-wise convolutions can provide position information is better than ordinary positional encoding.In the other branch,traditional convolutional neural networks(CNNs)are used to extract different features of fewer medical images.In addition,the attention fusion module BiFusion is used to effectively integrate the information from the CNN branch and Transformer branch,and the fused features can effectively capture the global and local context of the current spatial resolution.On the public standard datasets Gland Segmentation(GlaS),Colorectal adenocarcinoma gland(CRAG)and COVID-19 CT Images Segmentation,the F1-score,Intersection over Union(IoU)and parameters of the proposed TC-Fuse are superior to those by Axial Attention U-Net,U-Net,Medical Transformer and other methods.And F1-score increased respectively by 2.99%,3.42%and 3.95%compared with Medical Transformer.

A network lightweighting method for difficult segmentation of 3D medical images

Currently,deep learning is widely used in medical image segmentation and has achieved good results.However,3D medical image segmentation tasks with diverse lesion characters,blurred edges,and unstable positions require complex networks with a large number of parameters.It is computationally expensive and results in high requirements on equipment,making it hard to deploy the network in hospitals.In this work,we propose a method for network lightweighting and applied it to a 3D CNN based network.We experimented on a COVID-19 lesion segmentation dataset.Specifically,we use three cascaded one-dimensional convolutions to replace a 3D convolution,and integrate instance normalization with the previous layer of one-dimensional convolutions to accelerate network inference.In addition,we simplify test-time augmentation and deep supervision of the network.Experiments show that the lightweight network can reduce the prediction time of each sample and the memory usage by 50%and reduce the number of parameters by 60%compared with the original network.The training time of one epoch is also reduced by 50%with the segmentation accuracy dropped within the acceptable range.

AF-Net:A Medical Image Segmentation Network Based on Attention Mechanism and Feature Fusion

Medical image segmentation is an important application field of computer vision in medical image processing.Due to the close location and high similarity of different organs in medical images,the current segmentation algorithms have problems with mis-segmentation and poor edge segmentation.To address these challenges,we propose a medical image segmentation network(AF-Net)based on attention mechanism and feature fusion,which can effectively capture global information while focusing the network on the object area.In this approach,we add dual attention blocks(DA-block)to the backbone network,which comprises parallel channels and spatial attention branches,to adaptively calibrate and weigh features.Secondly,the multi-scale feature fusion block(MFF-block)is proposed to obtain feature maps of different receptive domains and get multi-scale information with less computational consumption.Finally,to restore the locations and shapes of organs,we adopt the global feature fusion blocks(GFF-block)to fuse high-level and low-level information,which can obtain accurate pixel positioning.We evaluate our method on multiple datasets(the aorta and lungs dataset),and the experimental results achieve 94.0%in mIoU and 96.3%in DICE,showing that our approach performs better than U-Net and other state-of-art methods.

Mu-Net:Multi-Path Upsampling Convolution Network for Medical Image Segmentation

Medical image segmentation plays an important role in clinical diagnosis,quantitative analysis,and treatment process.Since 2015,U-Net-based approaches have been widely used formedical image segmentation.The purpose of the U-Net expansive path is to map low-resolution encoder feature maps to full input resolution feature maps.However,the consecutive deconvolution and convolutional operations in the expansive path lead to the loss of some high-level information.More high-level information can make the segmentationmore accurate.In this paper,we propose MU-Net,a novel,multi-path upsampling convolution network to retain more high-level information.The MU-Net mainly consists of three parts:contracting path,skip connection,and multi-expansive paths.The proposed MU-Net architecture is evaluated based on three different medical imaging datasets.Our experiments show that MU-Net improves the segmentation performance of U-Net-based methods on different datasets.At the same time,the computational efficiency is significantly improved by reducing the number of parameters by more than half.

Improved Medical Image Segmentation Model Based on 3D U-Net

With the widespread application of deep learning in the field of computer vision,gradually allowing medical image technology to assist doctors in making diagnoses has great practical and research significance.Aiming at the shortcomings of the traditional U-Net model in 3D spatial information extraction,model over-fitting,and low degree of semantic information fusion,an improved medical image segmentation model has been used to achieve more accurate segmentation of medical images.In this model,we make full use of the residual network(ResNet)to solve the over-fitting problem.In order to process and aggregate data at different scales,the inception network is used instead of the traditional convolutional layer,and the dilated convolution is used to increase the receptive field.The conditional random field(CRF)can complete the contour refinement work.Compared with the traditional 3D U-Net network,the segmentation accuracy of the improved liver and tumor images increases by 2.89%and 7.66%,respectively.As a part of the image processing process,the method in this paper not only can be used for medical image segmentation,but also can lay the foundation for subsequent image 3D reconstruction work.

Automated Segmentation of the Brainstem,Cranial Nerves and Vessels for Trigeminal Neuralgia and Hemifacial Spasm

Accurate localization of cranial nerves and responsible blood vessels is important for diagnosing trigeminal neuralgia(TN)and hemifacial spasm(HFS).Manual delineation of the nerves and vessels on medical images is time-consuming and labor-intensive.Due to the development of convolutional neural networks(CNNs),the performance of medical image segmentation has been improved.In this work,we investigate the plans for automated segmentation of cranial nerves and responsible vessels for TN and HFS,which has not been comprehensively studied before.Different inputs are given to the CNN to find the best training configuration of segmenting trigeminal nerves,facial nerves,responsible vessels and brainstem,including the image modality and the number of segmentation targets.According to multiple experiments with seven training plans,we suggest training with the combination of three-dimensional fast imaging employing steady-state acquisition(3D-FIESTA)and three-dimensional time-of-flight magnetic resonance angiography(3DTOF-MRA),and separate segmentation of cranial nerves and vessels.

MEDICAL IMAGE SEGMENTATION BASED ON A MODIFIED LEVEL SET ALGORITHM

Objective To present a novel modified level set algorithm for medical image segmentation. Methods The algorithm is developed by substituting the speed function of level set algorithm with the region and gradient information of the image instead of the conventional gradient information. This new algorithm has been tested by a series of different modality medical images. Results We present various examples and also evaluate and compare the performance of our method with the classical level set method on weak boundaries and noisy images. Conclusion Experimental results show the proposed algorithm is effective and robust.

Designing a High-Performance Deep Learning Theoretical Model for Biomedical Image Segmentation by Using Key Elements of the Latest U-Net-Based Architectures

Deep learning (DL) has experienced an exponential development in recent years, with major impact in many medical fields, especially in the field of medical image and, respectively, as a specific task, in the segmentation of the medical image. We aim to create a computer assisted diagnostic method, optimized by the use of deep learning (DL) and validated by a randomized controlled clinical trial, is a highly automated tool for diagnosing and staging precancerous and cervical cancer and thyroid cancers. We aim to design a high-performance deep learning model, combined from convolutional neural network (U-Net)-based architectures, for segmentation of the medical image that is independent of the type of organs/tissues, dimensions or type of image (2D/3D) and to validate the DL model in a randomized, controlled clinical trial. We used as a methodology primarily the analysis of U-Net-based architectures to identify the key elements that we considered important in the design and optimization of the combined DL model, from the U-Net-based architectures, imagined by us. Secondly, we will validate the performance of the DL model through a randomized controlled clinical trial. The DL model designed by us will be a highly automated tool for diagnosing and staging precancers and cervical cancer and thyroid cancers. The combined model we designed takes into account the key features of each of the architectures Overcomplete Convolutional Network Kite-Net (Kite-Net), Attention gate mechanism is an improvement added on convolutional network architecture for fast and precise segmentation of images (Attention U-Net), Harmony Densely Connected Network-Medical image Segmentation (HarDNet-MSEG). In this regard, we will create a comprehensive computer assisted diagnostic methodology validated by a randomized controlled clinical trial. The model will be a highly automated tool for diagnosing and staging precancers and cervical cancer and thyroid cancers. This would help drastically minimize the time and effort that specialists put into analyzing medical images, help to achieve a better therapeutic plan, and can provide a “second opinion” of computer assisted diagnosis.

Intelligent Beetle Antenna Search with Deep Transfer Learning Enabled Medical Image Classification Model

Recently,computer assisted diagnosis(CAD)model creation has become more dependent on medical picture categorization.It is often used to identify several conditions,including brain disorders,diabetic retinopathy,and skin cancer.Most traditional CAD methods relied on textures,colours,and forms.Because many models are issue-oriented,they need a more substantial capacity to generalize and cannot capture high-level problem domain notions.Recent deep learning(DL)models have been published,providing a practical way to develop models specifically for classifying input medical pictures.This paper offers an intelligent beetle antenna search(IBAS-DTL)method for classifying medical images facilitated by deep transfer learning.The IBAS-DTL model aims to recognize and classify medical pictures into various groups.In order to segment medical pictures,the current IBASDTLM model first develops an entropy based weighting and first-order cumulative moment(EWFCM)approach.Additionally,the DenseNet-121 techniquewas used as a module for extracting features.ABASwith an extreme learning machine(ELM)model is used to classify the medical photos.A wide variety of tests were carried out using a benchmark medical imaging dataset to demonstrate the IBAS-DTL model’s noteworthy performance.The results gained indicated the IBAS-DTL model’s superiority over its pre-existing techniques.

Rethinking the Encoder-decoder Structure in Medical Image Segmentation from Releasing Decoder Structure

Medical image segmentation has witnessed rapid advancements with the emergence of encoder-decoder based methods.In the encoder-decoder structure,the primary goal of the decoding phase is not only to restore feature map resolution,but also to mitigate the loss of feature information incurred during the encoding phase.However,this approach gives rise to a challenge:multiple up-sampling operations in the decoder segment result in the loss of feature information.To address this challenge,we propose a novel network that removes the decoding structure to reduce feature information loss(CBL-Net).In particular,we introduce a Parallel Pooling Module(PPM)to counteract the feature information loss stemming from conventional and pooling operations during the encoding stage.Furthermore,we incorporate a Multiplexed Dilation Convolution(MDC)module to expand the network's receptive field.Also,although we have removed the decoding stage,we still need to recover the feature map resolution.Therefore,we introduced the Global Feature Recovery(GFR)module.It uses attention mechanism for the image feature map resolution recovery,which can effectively reduce the loss of feature information.We conduct extensive experimental evaluations on three publicly available medical image segmentation datasets:DRIVE,CHASEDB and MoNuSeg datasets.Experimental results show that our proposed network outperforms state-of-the-art methods in medical image segmentation.In addition,it achieves higher efficiency than the current network of coding and decoding structures by eliminating the decoding component.

Semantic Constraint Based Unsupervised Domain Adaptation for Cardiac Segmentation

The segmentation of unlabeled medical images is troublesome due to the high cost of annotation, and unsupervised domain adaptation is one solution to this. In this paper, an improved unsupervised domain adaptation method was proposed. The proposed method considered both global alignment and category-wise alignment. First, we aligned the appearance of two domains by image transformation. Second, we aligned the output maps of two domains in a global way. Then, we decomposed the semantic prediction map by category, aligning the prediction maps in a category-wise manner. Finally, we evaluated the proposed method on the 2017 Multi-Modality Whole Heart Segmentation Challenge dataset, and obtained 82.1 on the dice similarity coefficient and 4.6 on the average symmetric surface distance, demonstrating the effectiveness of the combination of global alignment and category-wise alignment.

Interactivemedical image segmentation with self-adaptive confidence calibration

Interactive medical image segmentation based on human-in-the-loop machine learning is a novel paradigm that draws on human expert knowledge to assist medical image segmentation.However,existing methods often fall into what we call interactive misunderstanding,the essence of which is the dilemma in trading off short-and long-term interaction information.To better use the interaction information at various timescales,we propose an interactive segmentation framework,called interactive MEdical image segmentation with self-adaptive Confidence CAlibration(MECCA),which combines action-based confidence learning and multi-agent reinforcement learning.A novel confidence network is learned by predicting the alignment level of the action with short-term interaction information.A confidence-based reward-shaping mechanism is then proposed to explicitly incorporate confidence in the policy gradient calculation,thus directly correcting the model’s interactive misunderstanding.MECCA also enables user-friendly interactions by reducing the interaction intensity and difficulty via label generation and interaction guidance,respectively.Numerical experiments on different segmentation tasks show that MECCA can significantly improve short-and long-term interaction information utilization efficiency with remarkably fewer labeled samples.The demo video is available at https://bit.ly/mecca-demo-video.

Inner Cascaded U^(2)-Net:An Improvement to Plain Cascaded U-Net

Deep neural networks are now widely used in the medical image segmentation field for their performance superiority and no need of manual feature extraction.U-Net has been the baseline model since the very beginning due to a symmetricalU-structure for better feature extraction and fusing and suitable for small datasets.To enhance the segmentation performance of U-Net,cascaded U-Net proposes to put two U-Nets successively to segment targets from coarse to fine.However,the plain cascaded U-Net faces the problem of too less between connections so the contextual information learned by the former U-Net cannot be fully used by the latter one.In this article,we devise novel Inner Cascaded U-Net and Inner Cascaded U^(2)-Net as improvements to plain cascaded U-Net for medical image segmentation.The proposed Inner Cascaded U-Net adds inner nested connections between two U-Nets to share more contextual information.To further boost segmentation performance,we propose Inner Cascaded U^(2)-Net,which applies residual U-block to capture more global contextual information from different scales.The proposed models can be trained from scratch in an end-to-end fashion and have been evaluated on Multimodal Brain Tumor Segmentation Challenge(BraTS)2013 and ISBI Liver Tumor Segmentation Challenge(LiTS)dataset in comparison to related U-Net,cascaded U-Net,U-Net++,U^(2)-Net and state-of-the-art methods.Our experiments demonstrate that our proposed Inner Cascaded U-Net and Inner Cascaded U^(2)-Net achieve better segmentation performance in terms of dice similarity coefficient and hausdorff distance as well as get finer outline segmentation.

Medical Image Segmentation using PCNN based on Multi-feature Grey Wolf Optimizer Bionic Algorithm

Medical image segmentation is a challenging task especially in multimodality medical image analysis.In this paper,an improved pulse coupled neural network based on multiple hybrid features grey wolf optimizer(MFGWO-PCNN)is proposed for multimodality medical image segmentation.Specifically,a two-stage medical image segmentation method based on bionic algorithm is presented,including image fusion and image segmentation.The image fusion stage fuses rich information from different modalities by utilizing a multimodality medical image fusion model based on maximum energy region.In the stage of image segmentation,an improved PCNN model based on MFGWO is proposed,which can adaptively set the parameters of PCNN according to the features of the image.Two modalities of FLAIR and TIC brain MRIs are applied to verify the effectiveness of the proposed MFGWO-PCNN algorithm.The experimental results demonstrate that the proposed method outperforms the other seven algorithms in subjective vision and objective evaluation indicators.

A Multiscale Approach to Automatic Medical Image Segmentation Using Self-Organizing Map

In this paper, a new medical image classification scheme is proposed using selforganizing map (SOM) combined with multiscale technique. It addresses the problem of the handling of edge pixels in the traditional multiscale SOM classifiers. First, to solve the difficulty in manual selection of edge pixels, a multiscale edge detection algorithm based on wavelet transform is proposed. Edge pixels detected are then selected into the training set as a new class and a mu1tiscale SoM classifier is trained using this training set. In this new scheme, the SoM classifier can perform both the classification on the entire image and the edge detection simultaneously. On the other hand, the misclassification of the traditional multiscale SoM classifier in regions near edges is greatly reduced and the correct classification is improved at the same time.

Automatic delineation of organs at risk in non-small cell lung cancer radiotherapy based on deep learning networks

Objective To introduce an end-to-end automatic segmentation method for organs at risk(OARs)in chest computed tomography(CT)images based on dense connection deep learning and to provide an accurate auto-segmentation model to reduce the workload on radiation oncologists.Methods CT images of 36 lung cancer cases were included in this study.Of these,27 cases were randomly selected as the training set,six cases as the validation set,and nine cases as the testing set.The left and right lungs,cord,and heart were auto-segmented,and the training time was set to approximately 5 h.The testing set was evaluated using geometric metrics including the Dice similarity coefficient(DSC),95%Hausdorff distance(HD95),and average surface distance(ASD).Thereafter,two sets of treatment plans were optimized based on manually contoured OARs and automatically contoured OARs,respectively.Dosimetric parameters including Dmax and Vx of the OARs were obtained and compared.Results The proposed model was superior to U-Net in terms of the DSC,HD95,and ASD,although there was no significant difference in the segmentation results yielded by both networks(P>0.05).Compared to manual segmentation,auto-segmentation significantly reduced the segmentation time by nearly 40.7%(P<0.05).Moreover,the differences in dose-volume parameters between the two sets of plans were not statistically significant(P>0.05).Conclusion The bilateral lung,cord,and heart could be accurately delineated using the DenseNet-based deep learning method.Thus,feature map reuse can be a novel approach to medical image auto-segmentation.