A review of medical ocular image segmentation

Deep learning has been extensively applied to medical image segmentation,resulting in significant advancements in the field of deep neural networks for medical image segmentation since the notable success of U Net in 2015.However,the application of deep learning models to ocular medical image segmentation poses unique challenges,especially compared to other body parts,due to the complexity,small size,and blurriness of such images,coupled with the scarcity of data.This article aims to provide a comprehensive review of medical image segmentation from two perspectives:the development of deep network structures and the application of segmentation in ocular imaging.Initially,the article introduces an overview of medical imaging,data processing,and performance evaluation metrics.Subsequently,it analyzes recent developments in U-Net-based network structures.Finally,for the segmentation of ocular medical images,the application of deep learning is reviewed and categorized by the type of ocular tissue.

DCFNet:An Effective Dual-Branch Cross-Attention Fusion Network for Medical Image Segmentation

Automatic segmentation of medical images provides a reliable scientific basis for disease diagnosis and analysis.Notably,most existing methods that combine the strengths of convolutional neural networks(CNNs)and Transformers have made significant progress.However,there are some limitations in the current integration of CNN and Transformer technology in two key aspects.Firstly,most methods either overlook or fail to fully incorporate the complementary nature between local and global features.Secondly,the significance of integrating the multiscale encoder features from the dual-branch network to enhance the decoding features is often disregarded in methods that combine CNN and Transformer.To address this issue,we present a groundbreaking dual-branch cross-attention fusion network(DCFNet),which efficiently combines the power of Swin Transformer and CNN to generate complementary global and local features.We then designed the Feature Cross-Fusion(FCF)module to efficiently fuse local and global features.In the FCF,the utilization of the Channel-wise Cross-fusion Transformer(CCT)serves the purpose of aggregatingmulti-scale features,and the Feature FusionModule(FFM)is employed to effectively aggregate dual-branch prominent feature regions from the spatial perspective.Furthermore,within the decoding phase of the dual-branch network,our proposed Channel Attention Block(CAB)aims to emphasize the significance of the channel features between the up-sampled features and the features generated by the FCFmodule to enhance the details of the decoding.Experimental results demonstrate that DCFNet exhibits enhanced accuracy in segmentation performance.Compared to other state-of-the-art(SOTA)methods,our segmentation framework exhibits a superior level of competitiveness.DCFNet’s accurate segmentation of medical images can greatly assist medical professionals in making crucial diagnoses of lesion areas in advance.

ATFF: Advanced Transformer with Multiscale Contextual Fusion for Medical Image Segmentation

Deep convolutional neural network (CNN) greatly promotes the automatic segmentation of medical images. However, due to the inherent properties of convolution operations, CNN usually cannot establish long-distance interdependence, which limits the segmentation performance. Transformer has been successfully applied to various computer vision, using self-attention mechanism to simulate long-distance interaction, so as to capture global information. However, self-attention lacks spatial location and high-performance computing. In order to solve the above problems, we develop a new medical transformer, which has a multi-scale context fusion function and can be used for medical image segmentation. The proposed model combines convolution operation and attention mechanism to form a u-shaped framework, which can capture both local and global information. First, the traditional converter module is improved to an advanced converter module, which uses post-layer normalization to obtain mild activation values, and uses scaled cosine attention with a moving window to obtain accurate spatial information. Secondly, we also introduce a deep supervision strategy to guide the model to fuse multi-scale feature information. It further enables the proposed model to effectively propagate feature information across layers, Thanks to this, it can achieve better segmentation performance while being more robust and efficient. The proposed model is evaluated on multiple medical image segmentation datasets. Experimental results demonstrate that the proposed model achieves better performance on a challenging dataset (ETIS) compared to existing methods that rely only on convolutional neural networks, transformers, or a combination of both. The mDice and mIou indicators increased by 2.74% and 3.3% respectively.

Two-Staged Method for Ice Channel Identification Based on Image Segmentation and Corner Point Regression

Identification of the ice channel is the basic technology for developing intelligent ships in ice-covered waters,which is important to ensure the safety and economy of navigation.In the Arctic,merchant ships with low ice class often navigate in channels opened up by icebreakers.Navigation in the ice channel often depends on good maneuverability skills and abundant experience from the captain to a large extent.The ship may get stuck if steered into ice fields off the channel.Under this circumstance,it is very important to study how to identify the boundary lines of ice channels with a reliable method.In this paper,a two-staged ice channel identification method is developed based on image segmentation and corner point regression.The first stage employs the image segmentation method to extract channel regions.In the second stage,an intelligent corner regression network is proposed to extract the channel boundary lines from the channel region.A non-intelligent angle-based filtering and clustering method is proposed and compared with corner point regression network.The training and evaluation of the segmentation method and corner regression network are carried out on the synthetic and real ice channel dataset.The evaluation results show that the accuracy of the method using the corner point regression network in the second stage is achieved as high as 73.33%on the synthetic ice channel dataset and 70.66%on the real ice channel dataset,and the processing speed can reach up to 14.58frames per second.

Multi-Level Image Segmentation Combining Chaotic Initialized Chimp Optimization Algorithm and Cauchy Mutation

To enhance the diversity and distribution uniformity of initial population,as well as to avoid local extrema in the Chimp Optimization Algorithm(CHOA),this paper improves the CHOA based on chaos initialization and Cauchy mutation.First,Sin chaos is introduced to improve the random population initialization scheme of the CHOA,which not only guarantees the diversity of the population,but also enhances the distribution uniformity of the initial population.Next,Cauchy mutation is added to optimize the global search ability of the CHOA in the process of position(threshold)updating to avoid the CHOA falling into local optima.Finally,an improved CHOA was formed through the combination of chaos initialization and Cauchy mutation(CICMCHOA),then taking fuzzy Kapur as the objective function,this paper applied CICMCHOA to natural and medical image segmentation,and compared it with four algorithms,including the improved Satin Bowerbird optimizer(ISBO),Cuckoo Search(ICS),etc.The experimental results deriving from visual and specific indicators demonstrate that CICMCHOA delivers superior segmentation effects in image segmentation.

Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Graph learning,when used as a semi-supervised learning(SSL)method,performs well for classification tasks with a low label rate.We provide a graph-based batch active learning pipeline for pixel/patch neighborhood multi-or hyperspectral image segmentation.Our batch active learning approach selects a collection of unlabeled pixels that satisfy a graph local maximum constraint for the active learning acquisition function that determines the relative importance of each pixel to the classification.This work builds on recent advances in the design of novel active learning acquisition functions(e.g.,the Model Change approach in arXiv:2110.07739)while adding important further developments including patch-neighborhood image analysis and batch active learning methods to further increase the accuracy and greatly increase the computational efficiency of these methods.In addition to improvements in the accuracy,our approach can greatly reduce the number of labeled pixels needed to achieve the same level of the accuracy based on randomly selected labeled pixels.

UDT:U-shaped deformable transformer for subarachnoid haemorrhage image segmentation

Subarachnoid haemorrhage(SAH),mostly caused by the rupture of intracranial aneu-rysm,is a common disease with a high fatality rate.SAH lesions are generally diffusely distributed,showing a variety of scales with irregular edges.The complex characteristics of lesions make SAH segmentation a challenging task.To cope with these difficulties,a u-shaped deformable transformer(UDT)is proposed for SAH segmentation.Specifically,first,a multi-scale deformable attention(MSDA)module is exploited to model the diffuseness and scale-variant characteristics of SAH lesions,where the MSDA module can fuse features in different scales and adjust the attention field of each element dynamically to generate discriminative multi-scale features.Second,the cross deformable attention-based skip connection(CDASC)module is designed to model the irregular edge char-acteristic of SAH lesions,where the CDASC module can utilise the spatial details from encoder features to refine the spatial information of decoder features.Third,the MSDA and CDASC modules are embedded into the backbone Res-UNet to construct the proposed UDT.Extensive experiments are conducted on the self-built SAH-CT dataset and two public medical datasets(GlaS and MoNuSeg).Experimental results show that the presented UDT achieves the state-of-the-art performance.

Transfer learning from T1-weighted to T2-weighted Magnetic resonance sequences for brain image segmentation

Magnetic resonance(MR)imaging is a widely employed medical imaging technique that produces detailed anatomical images of the human body.The segmentation of MR im-ages plays a crucial role in medical image analysis,as it enables accurate diagnosis,treatment planning,and monitoring of various diseases and conditions.Due to the lack of sufficient medical images,it is challenging to achieve an accurate segmentation,especially with the application of deep learning networks.The aim of this work is to study transfer learning from T1-weighted(T1-w)to T2-weighted(T2-w)MR sequences to enhance bone segmentation with minimal required computation resources.With the use of an excitation-based convolutional neural networks,four transfer learning mechanisms are proposed:transfer learning without fine tuning,open fine tuning,conservative fine tuning,and hybrid transfer learning.Moreover,a multi-parametric segmentation model is proposed using T2-w MR as an intensity-based augmentation technique.The novelty of this work emerges in the hybrid transfer learning approach that overcomes the overfitting issue and preserves the features of both modalities with minimal computation time and resources.The segmentation results are evaluated using 14 clinical 3D brain MR and CT images.The results reveal that hybrid transfer learning is superior for bone segmentation in terms of performance and computation time with DSCs of 0.5393±0.0007.Although T2-w-based augmentation has no significant impact on the performance of T1-w MR segmentation,it helps in improving T2-w MR segmentation and developing a multi-sequences segmentation model.

An Efficient Local Radial Basis Function Method for Image Segmentation Based on the Chan-Vese Model

In this paper,we consider the Chan–Vese(C-V)model for image segmentation and obtain its numerical solution accurately and efficiently.For this purpose,we present a local radial basis function method based on a Gaussian kernel(GA-LRBF)for spatial discretization.Compared to the standard radial basis functionmethod,this approach consumes less CPU time and maintains good stability because it uses only a small subset of points in the whole computational domain.Additionally,since the Gaussian function has the property of dimensional separation,the GA-LRBF method is suitable for dealing with isotropic images.Finally,a numerical scheme that couples GA-LRBF with the fourth-order Runge–Kutta method is applied to the C-V model,and a comparison of some numerical results demonstrates that this scheme achieves much more reliable image segmentation.

An Efficient Smoothing and Thresholding Image Segmentation Framework with Weighted Anisotropic-lsotropicTotalVariation

In this paper,we design an efficient,multi-stage image segmentation framework that incorporates a weighted difference of anisotropic and isotropic total variation(AITV).The segmentation framework generally consists of two stages:smoothing and thresholding,thus referred to as smoothing-and-thresholding(SaT).In the first stage,a smoothed image is obtained by an AITV-regularized Mumford-Shah(MS)model,which can be solved efficiently by the alternating direction method of multipliers(ADMMs)with a closed-form solution of a proximal operator of the l_(1)-αl_(2) regularizer.The convergence of the ADMM algorithm is analyzed.In the second stage,we threshold the smoothed image by K-means clustering to obtain the final segmentation result.Numerical experiments demonstrate that the proposed segmentation framework is versatile for both grayscale and color images,effcient in producing high-quality segmentation results within a few seconds,and robust to input images that are corrupted with noise,blur,or both.We compare the AITV method with its original convex TV and nonconvex TVP(O<p<1)counterparts,showcasing the qualitative and quantitative advantages of our proposed method.

Application of U-Net and Optimized Clustering in Medical Image Segmentation:A Review

As a mainstream research direction in the field of image segmentation,medical image segmentation plays a key role in the quantification of lesions,three-dimensional reconstruction,region of interest extraction and so on.Compared with natural images,medical images have a variety of modes.Besides,the emphasis of information which is conveyed by images of different modes is quite different.Because it is time-consuming and inefficient to manually segment medical images only by professional and experienced doctors.Therefore,large quantities of automated medical image segmentation methods have been developed.However,until now,researchers have not developed a universal method for all types of medical image segmentation.This paper reviews the literature on segmentation techniques that have produced major breakthroughs in recent years.Among the large quantities of medical image segmentation methods,this paper mainly discusses two categories of medical image segmentation methods.One is the improved strategies based on traditional clustering method.The other is the research progress of the improved image segmentation network structure model based on U-Net.The power of technology proves that the performance of the deep learning-based method is significantly better than that of the traditional method.This paper discussed both advantages and disadvantages of different algorithms and detailed how these methods can be used for the segmentation of lesions or other organs and tissues,as well as possible technical trends for future work.

TC-Fuse: A Transformers Fusing CNNs Network for Medical Image Segmentation

In medical image segmentation task,convolutional neural networks(CNNs)are difficult to capture long-range dependencies,but transformers can model the long-range dependencies effectively.However,transformers have a flexible structure and seldom assume the structural bias of input data,so it is difficult for transformers to learn positional encoding of the medical images when using fewer images for training.To solve these problems,a dual branch structure is proposed.In one branch,Mix-Feed-Forward Network(Mix-FFN)and axial attention are adopted to capture long-range dependencies and keep the translation invariance of the model.Mix-FFN whose depth-wise convolutions can provide position information is better than ordinary positional encoding.In the other branch,traditional convolutional neural networks(CNNs)are used to extract different features of fewer medical images.In addition,the attention fusion module BiFusion is used to effectively integrate the information from the CNN branch and Transformer branch,and the fused features can effectively capture the global and local context of the current spatial resolution.On the public standard datasets Gland Segmentation(GlaS),Colorectal adenocarcinoma gland(CRAG)and COVID-19 CT Images Segmentation,the F1-score,Intersection over Union(IoU)and parameters of the proposed TC-Fuse are superior to those by Axial Attention U-Net,U-Net,Medical Transformer and other methods.And F1-score increased respectively by 2.99%,3.42%and 3.95%compared with Medical Transformer.

Deep Learning for Image Segmentation: A Focus on Medical Imaging

Image segmentation is crucial for various research areas. Manycomputer vision applications depend on segmenting images to understandthe scene, such as autonomous driving, surveillance systems, robotics, andmedical imaging. With the recent advances in deep learning (DL) and itsconfounding results in image segmentation, more attention has been drawnto its use in medical image segmentation. This article introduces a surveyof the state-of-the-art deep convolution neural network (CNN) models andmechanisms utilized in image segmentation. First, segmentation models arecategorized based on their model architecture and primary working principle.Then, CNN categories are described, and various models are discussed withineach category. Compared with other existing surveys, several applicationswith multiple architectural adaptations are discussed within each category.A comparative summary is included to give the reader insights into utilizedarchitectures in different applications and datasets. This study focuses onmedical image segmentation applications, where the most widely used architecturesare illustrated, and other promising models are suggested that haveproven their success in different domains. Finally, the present work discussescurrent limitations and solutions along with future trends in the field.

Dual-Branch-UNet: A Dual-Branch Convolutional Neural Network for Medical Image Segmentation

In intelligent perception and diagnosis of medical equipment,the visual and morphological changes in retinal vessels are closely related to the severity of cardiovascular diseases(e.g.,diabetes and hypertension).Intelligent auxiliary diagnosis of these diseases depends on the accuracy of the retinal vascular segmentation results.To address this challenge,we design a Dual-Branch-UNet framework,which comprises a Dual-Branch encoder structure for feature extraction based on the traditional U-Net model for medical image segmentation.To be more explicit,we utilize a novel parallel encoder made up of various convolutional modules to enhance the encoder portion of the original U-Net.Then,image features are combined at each layer to produce richer semantic data and the model’s capacity is adjusted to various input images.Meanwhile,in the lower sampling section,we give up pooling and conduct the lower sampling by convolution operation to control step size for information fusion.We also employ an attentionmodule in the decoder stage to filter the image noises so as to lessen the response of irrelevant features.Experiments are verified and compared on the DRIVE and ARIA datasets for retinal vessels segmentation.The proposed Dual-Branch-UNet has proved to be superior to other five typical state-of-the-art methods.

LGNet:Local and global representation learning for fast biomedical image segmentation

Medical image segmentation plays a crucial role in clinical diagnosis and therapy systems,yet still faces many challenges.Building on convolutional neural networks(CNNs),medical image segmentation has achieved tremendous progress.However,owing to the locality of convolution operations,CNNs have the inherent limitation in learning global context.To address the limitation in building global context relationship from CNNs,we propose LGNet,a semantic segmentation network aiming to learn local and global features for fast and accurate medical image segmentation in this paper.Specifically,we employ a two-branch architecture consisting of convolution layers in one branch to learn local features and transformer layers in the other branch to learn global features.LGNet has two key insights:(1)We bridge two-branch to learn local and global features in an interactive way;(2)we present a novel multi-feature fusion model(MSFFM)to leverage the global contexture information from transformer and the local representational features from convolutions.Our method achieves state-of-the-art trade-off in terms of accuracy and efficiency on several medical image segmentation benchmarks including Synapse,ACDC and MOST.Specifically,LGNet achieves the state-of-the-art performance with Dice's indexes of 80.15%on Synapse,of 91.70%on ACDC,and of 95.56%on MOST.Meanwhile,the inference speed attains at 172 frames per second with 224-224 input resolution.The extensive experiments demonstrate the effectiveness of the proposed LGNet for fast and accurate for medical image segmentation.

An Improved Soft Subspace Clustering Algorithm for Brain MR Image Segmentation

In recent years,the soft subspace clustering algorithm has shown good results for high-dimensional data,which can assign different weights to each cluster class and use weights to measure the contribution of each dimension in various features.The enhanced soft subspace clustering algorithm combines interclass separation and intraclass tightness information,which has strong results for image segmentation,but the clustering algorithm is vulnerable to noisy data and dependence on the initialized clustering center.However,the clustering algorithmis susceptible to the influence of noisydata and reliance on initializedclustering centers andfalls into a local optimum;the clustering effect is poor for brain MR images with unclear boundaries and noise effects.To address these problems,a soft subspace clustering algorithm for brain MR images based on genetic algorithm optimization is proposed,which combines the generalized noise technique,relaxes the equational weight constraint in the objective function as the boundary constraint,and uses a genetic algorithm as a method to optimize the initialized clustering center.The genetic algorithm finds the best clustering center and reduces the algorithm’s dependence on the initial clustering center.The experiment verifies the robustness of the algorithm,as well as the noise immunity in various ways and shows good results on the common dataset and the brain MR images provided by the Changshu First People’s Hospital with specific high accuracy for clinical medicine.

An image segmentation method of pulverized coal for particle size analysis

An important index to evaluate the process efficiency of coal preparation is the mineral liberation degree of pulverized coal,which is greatly influenced by the particle size and shape distribution acquired by image segmentation.However,the agglomeration effect of fine powders and the edge effect of granular images caused by scanning electron microscopy greatly affect the precision of particle image segmentation.In this study,we propose a novel image segmentation method derived from mask regional convolutional neural network based on deep learning for recognizing fine coal powders.Firstly,an atrous convolution is introduced into our network to learn the image feature of multi-sized powders,which can reduce the missing segmentation of small-sized agglomerated particles.Then,a new mask loss function combing focal loss and dice coefficient is used to overcome the false segmentation caused by the edge effect.The final comparative experimental results show that our method achieves the best results of 94.43%and 91.44%on AP50 and AP75 respectively among the comparison algorithms.In addition,in order to provide an effective method for particle size analysis of coal particles,we study the particle size distribution of coal powders based on the proposed image segmentation method and obtain a good curve relationship between cumulative mass fraction and particle size.

Generative Deep Belief Model for Improved Medical Image Segmentation

Medical image assessment is based on segmentation at its fundamental stage.Deep neural networks have been more popular for segmentation work in recent years.However,the quality of labels has an impact on the training performance of these algorithms,particularly in the medical image domain,where both the interpretation cost and inter-observer variation are considerable.For this reason,a novel optimized deep learning approach is proposed for medical image segmentation.Optimization plays an important role in terms of resources used,accuracy,and the time taken.The noise in the raw medical image are processed using Quasi-Continuous Wavelet Transform(QCWT).Then,feature extraction and selection are done after the pre-processing of the image.The features are optimally selected by the Golden Eagle Optimization(GEO)method.Specifically,the processed image is segmented accurately using the proposed Generative Heap Belief Network(GHBN)technique.The execution of this research is done on MATLAB software.According to the results of the experiments,the proposed framework is superior to current techniques in terms of segmentation performance with a valid accuracy of 99%,which is comparable to the other methods.

Mobile-Deep Based PCB Image Segmentation Algorithm Research

Aiming at the problems of inaccurate edge segmentation,the hole phenomenon of segmenting large-scale targets,and the slow segmentation speed of printed circuit boards(PCB)in the image segmentation process,a PCB image segmentation model Mobile-Deep based on DeepLabv3+semantic segmentation framework is proposed.Firstly,the DeepLabv3+feature extraction network is replaced by the lightweight model MobileNetv2,which effectively reduces the number of model parameters;secondly,for the problem of positive and negative sample imbalance,a new loss function is composed of Focal Loss combined with Dice Loss to solve the category imbalance and improve the model discriminative ability;in addition,a more efficient atrous spatial pyramid pooling(E-ASPP)module is proposed.In addition,a more efficient E-ASPP module is proposed,and the Roberts crossover operator is chosen to sharpen the image edges to improve the model accuracy;finally,the network structure is redesigned to further improve the model accuracy by drawing on the multi-scale feature fusion approach.The experimental results show that the proposed segmentation algorithm achieves an average intersection ratio of 93.45%,a precision of 94.87%,a recall of 93.65%,and a balance score of 93.64%on the PCB test set,which is more accurate than the common segmentation algorithms Hrnetv2,UNet,PSPNet,and PCBSegClassNet,and the segmentation speed is faster.

Electrical Tree Image Segmentation Using Hybrid Multi Scale Line Tracking Algorithm

Electrical trees are an aging mechanismmost associated with partial discharge(PD)activities in crosslinked polyethylene(XLPE)insulation of high-voltage(HV)cables.Characterization of electrical tree structures gained considerable attention from researchers since a deep understanding of the tree morphology is required to develop new insulation material.Two-dimensional(2D)optical microscopy is primarily used to examine tree structures and propagation shapes with image segmentation methods.However,since electrical trees can emerge in different shapes such as bush-type or branch-type,treeing images are complicated to segment due to manifestation of convoluted tree branches,leading to a high misclassification rate during segmentation.Therefore,this study proposed a new method for segmenting 2D electrical tree images based on the multi-scale line tracking algorithm(MSLTA)by integrating batch processing method.The proposed method,h-MSLTA aims to provide accurate segmentation of electrical tree images obtained over a period of tree propagation observation under optical microscopy.The initial phase involves XLPE sample preparation and treeing image acquisition under real-time microscopy observation.The treeing images are then sampled and binarized in pre-processing.In the next phase,segmentation of tree structures is performed using the h-MSLTA by utilizing batch processing in multiple instances of treeing duration.Finally,the comparative investigation has been conducted using standard performance assessment metrics,including accuracy,sensitivity,specificity,Dice coefficient and Matthew’s correlation coefficient(MCC).Based on segmentation performance evaluation against several established segmentation methods,h-MSLTA achieved better results of 95.43%accuracy,97.28%specificity,69.43%sensitivity rate with 23.38%and 24.16%average improvement in Dice coefficient and MCC score respectively over the original algorithm.In addition,h-MSLTA produced accurate measurement results of global tree parameters of length and width in comparison with the ground truth image.These results indicated that the proposed method had a solid performance in terms of segmenting electrical tree branches in 2D treeing images compared to other established techniques.