Deep Learning for Image Segmentation: A Focus on Medical Imaging

Image segmentation is crucial for various research areas. Manycomputer vision applications depend on segmenting images to understandthe scene, such as autonomous driving, surveillance systems, robotics, andmedical imaging. With the recent advances in deep learning (DL) and itsconfounding results in image segmentation, more attention has been drawnto its use in medical image segmentation. This article introduces a surveyof the state-of-the-art deep convolution neural network (CNN) models andmechanisms utilized in image segmentation. First, segmentation models arecategorized based on their model architecture and primary working principle.Then, CNN categories are described, and various models are discussed withineach category. Compared with other existing surveys, several applicationswith multiple architectural adaptations are discussed within each category.A comparative summary is included to give the reader insights into utilizedarchitectures in different applications and datasets. This study focuses onmedical image segmentation applications, where the most widely used architecturesare illustrated, and other promising models are suggested that haveproven their success in different domains. Finally, the present work discussescurrent limitations and solutions along with future trends in the field.

An image segmentation based algorithm for imaging of slow slip earthquakes

Laboratory experiments next to a variety of observations, especially in subduction zones, have explored the existence of a premonitory stable slow slip growth phase preceding large earthquakes. These phe- nomena play an important role in the earthquake cycle and thus precise imaging and monitoring of these events are of great significance. In the literature, ENIF （extended network inversion filter） has been proposed as a rigorous algorithm capable of isolating signal from different types of noise and thereby provides us with deep insight into spatio-temporal evolution of slow slip events. Despite its considerable advantages, ENIF still suffers from some limitations. ENIF applies Tikhonov method of regularization with a quadratic form of cost function. While anomalous slip regions have clear contrast with the background slip in reality, Tikhonov regularization tends to over smooth （globally smooth） the slipping portion on the estimated images. In order to avoid over smoothing phenomenon, we have incorporated into ENIF an image segmentation step which tries to preserve edges of slow-slip event. As a second limitation, due to the nonlinearity imposed by such constraint as non-negativity of slip rate, uncertainty propagation through model is not simple. As the core of ENIF, EKF （extended Kalman filter）, performs uncertainty propagation by linearization of nonlinear model using Jacobian and Hessian matrices. As an alternative for EKE we have also investigated the application of UKF （unscented Kalman filter） which uses UT （unscented transform） for uncertainty propagation. Finally, we tested our proposed algorithm using a low signal to noise ratio synthetic data set. The results show a significant improvement in the perfor- mance of ENIF when the segmentation step is incorporated into the algorithm.

Two-Staged Method for Ice Channel Identification Based on Image Segmentation and Corner Point Regression

Identification of the ice channel is the basic technology for developing intelligent ships in ice-covered waters,which is important to ensure the safety and economy of navigation.In the Arctic,merchant ships with low ice class often navigate in channels opened up by icebreakers.Navigation in the ice channel often depends on good maneuverability skills and abundant experience from the captain to a large extent.The ship may get stuck if steered into ice fields off the channel.Under this circumstance,it is very important to study how to identify the boundary lines of ice channels with a reliable method.In this paper,a two-staged ice channel identification method is developed based on image segmentation and corner point regression.The first stage employs the image segmentation method to extract channel regions.In the second stage,an intelligent corner regression network is proposed to extract the channel boundary lines from the channel region.A non-intelligent angle-based filtering and clustering method is proposed and compared with corner point regression network.The training and evaluation of the segmentation method and corner regression network are carried out on the synthetic and real ice channel dataset.The evaluation results show that the accuracy of the method using the corner point regression network in the second stage is achieved as high as 73.33%on the synthetic ice channel dataset and 70.66%on the real ice channel dataset,and the processing speed can reach up to 14.58frames per second.

Improved organs at risk segmentation based on modified U‐Net with self‐attention and consistency regularisation

Cancer is one of the leading causes of death in the world,with radiotherapy as one of the treatment options.Radiotherapy planning starts with delineating the affected area from healthy organs,called organs at risk(OAR).A new approach to automatic OAR seg-mentation in the chest cavity in Computed Tomography(CT)images is presented.The proposed approach is based on the modified U‐Net architecture with the ResNet‐34 encoder,which is the baseline adopted in this work.The new two‐branch CS‐SA U‐Net architecture is proposed,which consists of two parallel U‐Net models in which self‐attention blocks with cosine similarity as query‐key similarity function(CS‐SA)blocks are inserted between the encoder and decoder,which enabled the use of con-sistency regularisation.The proposed solution demonstrates state‐of‐the‐art performance for the problem of OAR segmentation in CT images on the publicly available SegTHOR benchmark dataset in terms of a Dice coefficient(oesophagus-0.8714,heart-0.9516,trachea-0.9286,aorta-0.9510)and Hausdorff distance(oesophagus-0.2541,heart-0.1514,trachea-0.1722,aorta-0.1114)and significantly outperforms the baseline.The current approach is demonstrated to be viable for improving the quality of OAR segmentation for radiotherapy planning.

DCFNet:An Effective Dual-Branch Cross-Attention Fusion Network for Medical Image Segmentation

Automatic segmentation of medical images provides a reliable scientific basis for disease diagnosis and analysis.Notably,most existing methods that combine the strengths of convolutional neural networks(CNNs)and Transformers have made significant progress.However,there are some limitations in the current integration of CNN and Transformer technology in two key aspects.Firstly,most methods either overlook or fail to fully incorporate the complementary nature between local and global features.Secondly,the significance of integrating the multiscale encoder features from the dual-branch network to enhance the decoding features is often disregarded in methods that combine CNN and Transformer.To address this issue,we present a groundbreaking dual-branch cross-attention fusion network(DCFNet),which efficiently combines the power of Swin Transformer and CNN to generate complementary global and local features.We then designed the Feature Cross-Fusion(FCF)module to efficiently fuse local and global features.In the FCF,the utilization of the Channel-wise Cross-fusion Transformer(CCT)serves the purpose of aggregatingmulti-scale features,and the Feature FusionModule(FFM)is employed to effectively aggregate dual-branch prominent feature regions from the spatial perspective.Furthermore,within the decoding phase of the dual-branch network,our proposed Channel Attention Block(CAB)aims to emphasize the significance of the channel features between the up-sampled features and the features generated by the FCFmodule to enhance the details of the decoding.Experimental results demonstrate that DCFNet exhibits enhanced accuracy in segmentation performance.Compared to other state-of-the-art(SOTA)methods,our segmentation framework exhibits a superior level of competitiveness.DCFNet’s accurate segmentation of medical images can greatly assist medical professionals in making crucial diagnoses of lesion areas in advance.

Multi-Level Image Segmentation Combining Chaotic Initialized Chimp Optimization Algorithm and Cauchy Mutation

To enhance the diversity and distribution uniformity of initial population,as well as to avoid local extrema in the Chimp Optimization Algorithm(CHOA),this paper improves the CHOA based on chaos initialization and Cauchy mutation.First,Sin chaos is introduced to improve the random population initialization scheme of the CHOA,which not only guarantees the diversity of the population,but also enhances the distribution uniformity of the initial population.Next,Cauchy mutation is added to optimize the global search ability of the CHOA in the process of position(threshold)updating to avoid the CHOA falling into local optima.Finally,an improved CHOA was formed through the combination of chaos initialization and Cauchy mutation(CICMCHOA),then taking fuzzy Kapur as the objective function,this paper applied CICMCHOA to natural and medical image segmentation,and compared it with four algorithms,including the improved Satin Bowerbird optimizer(ISBO),Cuckoo Search(ICS),etc.The experimental results deriving from visual and specific indicators demonstrate that CICMCHOA delivers superior segmentation effects in image segmentation.

An Efficient Local Radial Basis Function Method for Image Segmentation Based on the Chan-Vese Model

In this paper,we consider the Chan–Vese(C-V)model for image segmentation and obtain its numerical solution accurately and efficiently.For this purpose,we present a local radial basis function method based on a Gaussian kernel(GA-LRBF)for spatial discretization.Compared to the standard radial basis functionmethod,this approach consumes less CPU time and maintains good stability because it uses only a small subset of points in the whole computational domain.Additionally,since the Gaussian function has the property of dimensional separation,the GA-LRBF method is suitable for dealing with isotropic images.Finally,a numerical scheme that couples GA-LRBF with the fourth-order Runge–Kutta method is applied to the C-V model,and a comparison of some numerical results demonstrates that this scheme achieves much more reliable image segmentation.

Guided-YNet: Saliency Feature-Guided Interactive Feature Enhancement Lung Tumor Segmentation Network

Multimodal lung tumor medical images can provide anatomical and functional information for the same lesion.Such as Positron Emission Computed Tomography(PET),Computed Tomography(CT),and PET-CT.How to utilize the lesion anatomical and functional information effectively and improve the network segmentation performance are key questions.To solve the problem,the Saliency Feature-Guided Interactive Feature Enhancement Lung Tumor Segmentation Network(Guide-YNet)is proposed in this paper.Firstly,a double-encoder single-decoder U-Net is used as the backbone in this model,a single-coder single-decoder U-Net is used to generate the saliency guided feature using PET image and transmit it into the skip connection of the backbone,and the high sensitivity of PET images to tumors is used to guide the network to accurately locate lesions.Secondly,a Cross Scale Feature Enhancement Module(CSFEM)is designed to extract multi-scale fusion features after downsampling.Thirdly,a Cross-Layer Interactive Feature Enhancement Module(CIFEM)is designed in the encoder to enhance the spatial position information and semantic information.Finally,a Cross-Dimension Cross-Layer Feature Enhancement Module(CCFEM)is proposed in the decoder,which effectively extractsmultimodal image features through global attention and multi-dimension local attention.The proposed method is verified on the lung multimodal medical image datasets,and the results showthat theMean Intersection overUnion(MIoU),Accuracy(Acc),Dice Similarity Coefficient(Dice),Volumetric overlap error(Voe),Relative volume difference(Rvd)of the proposed method on lung lesion segmentation are 87.27%,93.08%,97.77%,95.92%,89.28%,and 88.68%,respectively.It is of great significance for computer-aided diagnosis.

Semantic segmentation via pixel-to-center similarity calculation

Since the fully convolutional network has achieved great success in semantic segmentation,lots of works have been proposed to extract discriminative pixel representations.However,the authors observe that existing methods still suffer from two typical challenges:(i)The intra-class feature variation between different scenes may be large,leading to the difficulty in maintaining the consistency between same-class pixels from different scenes;(ii)The inter-class feature distinction in the same scene could be small,resulting in the limited performance to distinguish different classes in each scene.The authors first rethink se-mantic segmentation from a perspective of similarity between pixels and class centers.Each weight vector of the segmentation head represents its corresponding semantic class in the whole dataset,which can be regarded as the embedding of the class center.Thus,the pixel-wise classification amounts to computing similarity in the final feature space between pixels and the class centers.Under this novel view,the authors propose a Class Center Similarity(CCS)layer to address the above-mentioned challenges by generating adaptive class centers conditioned on each scenes and supervising the similarities between class centers.The CCS layer utilises the Adaptive Class Center Module to generate class centers conditioned on each scene,which adapt the large intra-class variation between different scenes.Specially designed Class Distance Loss(CD Loss)is introduced to control both inter-class and intra-class distances based on the predicted center-to-center and pixel-to-center similarity.Finally,the CCS layer outputs the processed pixel-to-center similarity as the segmentation prediction.Extensive experiments demonstrate that our model performs favourably against the state-of-the-art methods.

Transfer learning from T1-weighted to T2-weighted Magnetic resonance sequences for brain image segmentation

Magnetic resonance(MR)imaging is a widely employed medical imaging technique that produces detailed anatomical images of the human body.The segmentation of MR im-ages plays a crucial role in medical image analysis,as it enables accurate diagnosis,treatment planning,and monitoring of various diseases and conditions.Due to the lack of sufficient medical images,it is challenging to achieve an accurate segmentation,especially with the application of deep learning networks.The aim of this work is to study transfer learning from T1-weighted(T1-w)to T2-weighted(T2-w)MR sequences to enhance bone segmentation with minimal required computation resources.With the use of an excitation-based convolutional neural networks,four transfer learning mechanisms are proposed:transfer learning without fine tuning,open fine tuning,conservative fine tuning,and hybrid transfer learning.Moreover,a multi-parametric segmentation model is proposed using T2-w MR as an intensity-based augmentation technique.The novelty of this work emerges in the hybrid transfer learning approach that overcomes the overfitting issue and preserves the features of both modalities with minimal computation time and resources.The segmentation results are evaluated using 14 clinical 3D brain MR and CT images.The results reveal that hybrid transfer learning is superior for bone segmentation in terms of performance and computation time with DSCs of 0.5393±0.0007.Although T2-w-based augmentation has no significant impact on the performance of T1-w MR segmentation,it helps in improving T2-w MR segmentation and developing a multi-sequences segmentation model.

Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Graph learning,when used as a semi-supervised learning(SSL)method,performs well for classification tasks with a low label rate.We provide a graph-based batch active learning pipeline for pixel/patch neighborhood multi-or hyperspectral image segmentation.Our batch active learning approach selects a collection of unlabeled pixels that satisfy a graph local maximum constraint for the active learning acquisition function that determines the relative importance of each pixel to the classification.This work builds on recent advances in the design of novel active learning acquisition functions(e.g.,the Model Change approach in arXiv:2110.07739)while adding important further developments including patch-neighborhood image analysis and batch active learning methods to further increase the accuracy and greatly increase the computational efficiency of these methods.In addition to improvements in the accuracy,our approach can greatly reduce the number of labeled pixels needed to achieve the same level of the accuracy based on randomly selected labeled pixels.

An Efficient Smoothing and Thresholding Image Segmentation Framework with Weighted Anisotropic-Isotropic Total Variation

In this paper,we design an efficient,multi-stage image segmentation framework that incorporates a weighted difference of anisotropic and isotropic total variation(AITV).The segmentation framework generally consists of two stages:smoothing and thresholding,thus referred to as smoothing-and-thresholding(SaT).In the first stage,a smoothed image is obtained by an AITV-regularized Mumford-Shah(MS)model,which can be solved efficiently by the alternating direction method of multipliers(ADMMs)with a closed-form solution of a proximal operator of the l_(1)-αl_(2) regularizer.The convergence of the ADMM algorithm is analyzed.In the second stage,we threshold the smoothed image by K-means clustering to obtain the final segmentation result.Numerical experiments demonstrate that the proposed segmentation framework is versatile for both grayscale and color images,effcient in producing high-quality segmentation results within a few seconds,and robust to input images that are corrupted with noise,blur,or both.We compare the AITV method with its original convex TV and nonconvex TVP(O<p<1)counterparts,showcasing the qualitative and quantitative advantages of our proposed method.

Enhancing Building Facade Image Segmentation via Object-Wise Processing and Cascade U-Net

The growing demand for energy-efficient solutions has led to increased interest in analyzing building facades,as buildings contribute significantly to energy consumption in urban environments.However,conventional image segmentation methods often struggle to capture fine details such as edges and contours,limiting their effectiveness in identifying areas prone to energy loss.To address this challenge,we propose a novel segmentation methodology that combines object-wise processing with a two-stage deep learning model,Cascade U-Net.Object-wise processing isolates components of the facade,such as walls and windows,for independent analysis,while Cascade U-Net incorporates contour information to enhance segmentation accuracy.The methodology involves four steps:object isolation,which crops and adjusts the image based on bounding boxes;contour extraction,which derives contours;image segmentation,which modifies and reuses contours as guide data in Cascade U-Net to segment areas;and segmentation synthesis,which integrates the results obtained for each object to produce the final segmentation map.Applied to a dataset of Korean building images,the proposed method significantly outperformed traditional models,demonstrating improved accuracy and the ability to preserve critical structural details.Furthermore,we applied this approach to classify window thermal loss in real-world scenarios using infrared images,showing its potential to identify windows vulnerable to energy loss.Notably,our Cascade U-Net,which builds upon the relatively lightweight U-Net architecture,also exhibited strong performance,reinforcing the practical value of this method.Our approach offers a practical solution for enhancing energy efficiency in buildings by providing more precise segmentation results.

A review of medical ocular image segmentation

Deep learning has been extensively applied to medical image segmentation,resulting in significant advancements in the field of deep neural networks for medical image segmentation since the notable success of U Net in 2015.However,the application of deep learning models to ocular medical image segmentation poses unique challenges,especially compared to other body parts,due to the complexity,small size,and blurriness of such images,coupled with the scarcity of data.This article aims to provide a comprehensive review of medical image segmentation from two perspectives:the development of deep network structures and the application of segmentation in ocular imaging.Initially,the article introduces an overview of medical imaging,data processing,and performance evaluation metrics.Subsequently,it analyzes recent developments in U-Net-based network structures.Finally,for the segmentation of ocular medical images,the application of deep learning is reviewed and categorized by the type of ocular tissue.

UDT:U-shaped deformable transformer for subarachnoid haemorrhage image segmentation

Subarachnoid haemorrhage(SAH),mostly caused by the rupture of intracranial aneu-rysm,is a common disease with a high fatality rate.SAH lesions are generally diffusely distributed,showing a variety of scales with irregular edges.The complex characteristics of lesions make SAH segmentation a challenging task.To cope with these difficulties,a u-shaped deformable transformer(UDT)is proposed for SAH segmentation.Specifically,first,a multi-scale deformable attention(MSDA)module is exploited to model the diffuseness and scale-variant characteristics of SAH lesions,where the MSDA module can fuse features in different scales and adjust the attention field of each element dynamically to generate discriminative multi-scale features.Second,the cross deformable attention-based skip connection(CDASC)module is designed to model the irregular edge char-acteristic of SAH lesions,where the CDASC module can utilise the spatial details from encoder features to refine the spatial information of decoder features.Third,the MSDA and CDASC modules are embedded into the backbone Res-UNet to construct the proposed UDT.Extensive experiments are conducted on the self-built SAH-CT dataset and two public medical datasets(GlaS and MoNuSeg).Experimental results show that the presented UDT achieves the state-of-the-art performance.

ATFF: Advanced Transformer with Multiscale Contextual Fusion for Medical Image Segmentation

Deep convolutional neural network (CNN) greatly promotes the automatic segmentation of medical images. However, due to the inherent properties of convolution operations, CNN usually cannot establish long-distance interdependence, which limits the segmentation performance. Transformer has been successfully applied to various computer vision, using self-attention mechanism to simulate long-distance interaction, so as to capture global information. However, self-attention lacks spatial location and high-performance computing. In order to solve the above problems, we develop a new medical transformer, which has a multi-scale context fusion function and can be used for medical image segmentation. The proposed model combines convolution operation and attention mechanism to form a u-shaped framework, which can capture both local and global information. First, the traditional converter module is improved to an advanced converter module, which uses post-layer normalization to obtain mild activation values, and uses scaled cosine attention with a moving window to obtain accurate spatial information. Secondly, we also introduce a deep supervision strategy to guide the model to fuse multi-scale feature information. It further enables the proposed model to effectively propagate feature information across layers, Thanks to this, it can achieve better segmentation performance while being more robust and efficient. The proposed model is evaluated on multiple medical image segmentation datasets. Experimental results demonstrate that the proposed model achieves better performance on a challenging dataset (ETIS) compared to existing methods that rely only on convolutional neural networks, transformers, or a combination of both. The mDice and mIou indicators increased by 2.74% and 3.3% respectively.

A Review of Research on Accurate Segmentation of Multimodal Tumor Images

Accurate segmentation of tumor images is a key core technology for the diagnosis and treatment of tumor diseases.In this paper,we analyze a variety of novel and targeted algorithms to solve these problems,summarize,and elaborate the method based on multimodal tumor image processing given the characteristics of serious grayscale inhomogeneity,texture instability,and diversity complexity of tumor images.

Segmentation of scattered point data through a new curvature analysis algorithm

A systematic scheme is proposed to automatically extract geometric surface features from a point cloud composed of a set of unorganized three-dimensional coordinate points by data segmentation. The key technology is a algorithm that estimates the local surface curvature properties of scattered point data based on local base surface parameterization. Eight surface types from the signs of the Gaussian and mean curvatures provide an initial segmentation, which will be refined by an iterative region growing method. Experimental results show the scheme's performance on two point clouds.

A New Adaptive Image Segmentation Method

Aim Researching the optimal thieshold of image segmentation. M^ethods An adaptiveimages segmentation method based on the entropy of histogram of gray-level picture and genetic. algorithm (GA) was presental. Results In our approach, the segmentation problem was formulated as an optimization problem and the fitness of GA which can efficiently search the segmentation parameter space was regarded as the quality criterion. Conclusion The methodcan be adapted for optimal behold segmentation.

EFFECTIVE IMAGE SEGMENTATION FRAMEWORK FOR GAUSSIAN MIXTURE MODEL INCORPORATING LOCAL INFORMATION

A new two-step framework is proposed for image segmentation. In the first step, the gray-value distribution of the given image is reshaped to have larger inter-class variance and less intra-class variance. In the sec- ond step, the discriminant-based methods or clustering-based methods are performed on the reformed distribution. It is focused on the typical clustering methods-Gaussian mixture model （GMM） and its variant to demonstrate the feasibility of the framework. Due to the independence of the first step in its second step, it can be integrated into the pixel-based and the histogram-based methods to improve their segmentation quality. The experiments on artificial and real images show that the framework can achieve effective and robust segmentation results.