Axial Assembled Correspondence Network for Few-Shot Semantic Segmentation

Few-shot semantic segmentation aims at training a model that can segment novel classes in a query image with only a few densely annotated support exemplars.It remains a challenge because of large intra-class variations between the support and query images.Existing approaches utilize 4D convolutions to mine semantic correspondence between the support and query images.However,they still suffer from heavy computation,sparse correspondence,and large memory.We propose axial assembled correspondence network(AACNet)to alleviate these issues.The key point of AACNet is the proposed axial assembled 4D kernel,which constructs the basic block for semantic correspondence encoder(SCE).Furthermore,we propose the deblurring equations to provide more robust correspondence for the aforementioned SCE and design a novel fusion module to mix correspondences in a learnable manner.Experiments on PASCAL-5~i reveal that our AACNet achieves a mean intersection-over-union score of 65.9%for 1-shot segmentation and 70.6%for 5-shot segmentation,surpassing the state-of-the-art method by 5.8%and 5.0%respectively.

Part-Whole Relational Few-Shot 3D Point Cloud Semantic Segmentation

This paper focuses on the task of few-shot 3D point cloud semantic segmentation.Despite some progress,this task still encounters many issues due to the insufficient samples given,e.g.,incomplete object segmentation and inaccurate semantic discrimination.To tackle these issues,we first leverage part-whole relationships into the task of 3D point cloud semantic segmentation to capture semantic integrity,which is empowered by the dynamic capsule routing with the module of 3D Capsule Networks(CapsNets)in the embedding network.Concretely,the dynamic routing amalgamates geometric information of the 3D point cloud data to construct higher-level feature representations,which capture the relationships between object parts and their wholes.Secondly,we designed a multi-prototype enhancement module to enhance the prototype discriminability.Specifically,the single-prototype enhancement mechanism is expanded to the multi-prototype enhancement version for capturing rich semantics.Besides,the shot-correlation within the category is calculated via the interaction of different samples to enhance the intra-category similarity.Ablation studies prove that the involved part-whole relations and proposed multi-prototype enhancement module help to achieve complete object segmentation and improve semantic discrimination.Moreover,under the integration of these two modules,quantitative and qualitative experiments on two public benchmarks,including S3DIS and ScanNet,indicate the superior performance of the proposed framework on the task of 3D point cloud semantic segmentation,compared to some state-of-the-art methods.

CrossFormer Embedding DeepLabv3+ for Remote Sensing Images Semantic Segmentation

High-resolution remote sensing image segmentation is a challenging task. In urban remote sensing, the presenceof occlusions and shadows often results in blurred or invisible object boundaries, thereby increasing the difficultyof segmentation. In this paper, an improved network with a cross-region self-attention mechanism for multi-scalefeatures based onDeepLabv3+is designed to address the difficulties of small object segmentation and blurred targetedge segmentation. First,we use CrossFormer as the backbone feature extraction network to achieve the interactionbetween large- and small-scale features, and establish self-attention associations between features at both large andsmall scales to capture global contextual feature information. Next, an improved atrous spatial pyramid poolingmodule is introduced to establish multi-scale feature maps with large- and small-scale feature associations, andattention vectors are added in the channel direction to enable adaptive adjustment of multi-scale channel features.The proposed networkmodel is validated using the PotsdamandVaihingen datasets. The experimental results showthat, compared with existing techniques, the network model designed in this paper can extract and fuse multiscaleinformation, more clearly extract edge information and small-scale information, and segment boundariesmore smoothly. Experimental results on public datasets demonstrate the superiority of ourmethod compared withseveral state-of-the-art networks.

A semantic segmentation-based underwater acoustic image transmission framework for cooperative SLAM

With the development of underwater sonar detection technology,simultaneous localization and mapping(SLAM)approach has attracted much attention in underwater navigation field in recent years.But the weak detection ability of a single vehicle limits the SLAM performance in wide areas.Thereby,cooperative SLAM using multiple vehicles has become an important research direction.The key factor of cooperative SLAM is timely and efficient sonar image transmission among underwater vehicles.However,the limited bandwidth of underwater acoustic channels contradicts a large amount of sonar image data.It is essential to compress the images before transmission.Recently,deep neural networks have great value in image compression by virtue of the powerful learning ability of neural networks,but the existing sonar image compression methods based on neural network usually focus on the pixel-level information without the semantic-level information.In this paper,we propose a novel underwater acoustic transmission scheme called UAT-SSIC that includes semantic segmentation-based sonar image compression(SSIC)framework and the joint source-channel codec,to improve the accuracy of the semantic information of the reconstructed sonar image at the receiver.The SSIC framework consists of Auto-Encoder structure-based sonar image compression network,which is measured by a semantic segmentation network's residual.Considering that sonar images have the characteristics of blurred target edges,the semantic segmentation network used a special dilated convolution neural network(DiCNN)to enhance segmentation accuracy by expanding the range of receptive fields.The joint source-channel codec with unequal error protection is proposed that adjusts the power level of the transmitted data,which deal with sonar image transmission error caused by the serious underwater acoustic channel.Experiment results demonstrate that our method preserves more semantic information,with advantages over existing methods at the same compression ratio.It also improves the error tolerance and packet loss resistance of transmission.

A Random Fusion of Mix 3D and Polar Mix to Improve Semantic Segmentation Performance in 3D Lidar Point Cloud

This paper focuses on the effective utilization of data augmentation techniques for 3Dlidar point clouds to enhance the performance of neural network models.These point clouds,which represent spatial information through a collection of 3D coordinates,have found wide-ranging applications.Data augmentation has emerged as a potent solution to the challenges posed by limited labeled data and the need to enhance model generalization capabilities.Much of the existing research is devoted to crafting novel data augmentation methods specifically for 3D lidar point clouds.However,there has been a lack of focus on making the most of the numerous existing augmentation techniques.Addressing this deficiency,this research investigates the possibility of combining two fundamental data augmentation strategies.The paper introduces PolarMix andMix3D,two commonly employed augmentation techniques,and presents a new approach,named RandomFusion.Instead of using a fixed or predetermined combination of augmentation methods,RandomFusion randomly chooses one method from a pool of options for each instance or sample.This innovative data augmentation technique randomly augments each point in the point cloud with either PolarMix or Mix3D.The crux of this strategy is the random choice between PolarMix and Mix3Dfor the augmentation of each point within the point cloud data set.The results of the experiments conducted validate the efficacy of the RandomFusion strategy in enhancing the performance of neural network models for 3D lidar point cloud semantic segmentation tasks.This is achieved without compromising computational efficiency.By examining the potential of merging different augmentation techniques,the research contributes significantly to a more comprehensive understanding of how to utilize existing augmentation methods for 3D lidar point clouds.RandomFusion data augmentation technique offers a simple yet effective method to leverage the diversity of augmentation techniques and boost the robustness of models.The insights gained from this research can pave the way for future work aimed at developing more advanced and efficient data augmentation strategies for 3D lidar point cloud analysis.

Industry-Oriented Detection Method of PCBA Defects Using Semantic Segmentation Models

Automated optical inspection(AOI)is a significant process in printed circuit board assembly(PCBA)production lines which aims to detect tiny defects in PCBAs.Existing AOI equipment has several deficiencies including low throughput,large computation cost,high latency,and poor flexibility,which limits the efficiency of online PCBA inspection.In this paper,a novel PCBA defect detection method based on a lightweight deep convolution neural network is proposed.In this method,the semantic segmentation model is combined with a rule-based defect recognition algorithm to build up a defect detection frame-work.To improve the performance of the model,extensive real PCBA images are collected from production lines as datasets.Some optimization methods have been applied in the model according to production demand and enable integration in lightweight computing devices.Experiment results show that the production line using our method realizes a throughput more than three times higher than traditional methods.Our method can be integrated into a lightweight inference system and pro-mote the flexibility of AOI.The proposed method builds up a general paradigm and excellent example for model design and optimization oriented towards industrial requirements.

Semantic segmentation of pyramidal neuron skeletons using geometric deep learning

Neurons can be abstractly represented as skeletons due to the filament nature of neurites.With the rapid development of imaging and image analysis techniques,an increasing amount of neuron skeleton data is being produced.In some scienti fic studies,it is necessary to dissect the axons and dendrites,which is typically done manually and is both tedious and time-consuming.To automate this process,we have developed a method that relies solely on neuronal skeletons using Geometric Deep Learning(GDL).We demonstrate the effectiveness of this method using pyramidal neurons in mammalian brains,and the results are promising for its application in neuroscience studies.

DuFNet:Dual Flow Network of Real-Time Semantic Segmentation for Unmanned Driving Application of Internet of Things

The application of unmanned driving in the Internet of Things is one of the concrete manifestations of the application of artificial intelligence technology.Image semantic segmentation can help the unmanned driving system by achieving road accessibility analysis.Semantic segmentation is also a challenging technology for image understanding and scene parsing.We focused on the challenging task of real-time semantic segmentation in this paper.In this paper,we proposed a novel fast architecture for real-time semantic segmentation named DuFNet.Starting from the existing work of Bilateral Segmentation Network(BiSeNet),DuFNet proposes a novel Semantic Information Flow(SIF)structure for context information and a novel Fringe Information Flow(FIF)structure for spatial information.We also proposed two kinds of SIF with cascaded and paralleled structures,respectively.The SIF encodes the input stage by stage in the ResNet18 backbone and provides context information for the feature fusionmodule.Features from previous stages usually contain rich low-level details but high-level semantics for later stages.Themultiple convolutions embed in Parallel SIF aggregate the corresponding features among different stages and generate a powerful global context representation with less computational cost.The FIF consists of a pooling layer and an upsampling operator followed by projection convolution layer.The concise component provides more spatial details for the network.Compared with BiSeNet,our work achieved faster speed and comparable performance with 72.34%mIoU accuracy and 78 FPS on Cityscapes Dataset based on the ResNet18 backbone.

Multi-task Learning of Semantic Segmentation and Height Estimation for Multi-modal Remote Sensing Images

Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.

Point Cloud Based Semantic Segmentation Method for Unmanned Shuttle Bus

The complexity of application scenarios and the enormous volume of point cloud data make it difficult to quickly and effectively segment the scenario only based on the point cloud.In this paper,to address the semantic segmentation for safety driving of unmanned shuttle buses,an accurate and effective point cloud-based semantic segmentation method is proposed for specified scenarios(such as campus).Firstly,we analyze the characteristic of the shuttle bus scenarios and propose to use ROI selection to reduce the total points in computation,and then propose an improved semantic segmentation model based on Cylinder3D,which improves mean Intersection over Union(mIoU)by 1.3%over the original model on SemanticKITTI data;then,a semantic category division method is proposed for road scenario of shuttle bus and practical application requirements,and then we further simplify the model to improve the efficiency without losing the accuracy.Finally,the nuScenes dataset and the real gathered campus scene data are used to validate and analyze the proposed method.The experimental results on the nuScenes dataset and our data demonstrate that the proposed method performs better than other point cloud semantic segmentation methods in terms of application requirements for unmanned shuttle buses.Which has a higher accuracy(82.73%in mIoU)and a higher computational efficiency(inference speed of 90 ms).

Bilateral U-Net semantic segmentation with spatial attention mechanism

Aiming at the problem that the existing models have a poor segmentation effect on imbalanced data sets with small-scale samples,a bilateral U-Net network model with a spatial attention mechanism is designed.The model uses the lightweight MobileNetV2 as the backbone network for feature hierarchical extraction and proposes an Attentive Pyramid Spatial Attention(APSA)module compared to the Attenuated Spatial Pyramid module,which can increase the receptive field and enhance the information,and finally adds the context fusion prediction branch that fuses high-semantic and low-semantic prediction results,and the model effectively improves the segmentation accuracy of small data sets.The experimental results on the CamVid data set show that compared with some existing semantic segmentation networks,the algorithm has a better segmentation effect and segmentation accuracy,and its mIOU reaches 75.85%.Moreover,to verify the generality of the model and the effectiveness of the APSA module,experiments were conducted on the VOC 2012 data set,and the APSA module improved mIOU by about 12.2%.

Semantic Segmentation by Using Down-Sampling and Subpixel Convolution: DSSC-UNet

Recently, semantic segmentation has been widely applied toimage processing, scene understanding, and many others. Especially, indeep learning-based semantic segmentation, the U-Net with convolutionalencoder-decoder architecture is a representative model which is proposed forimage segmentation in the biomedical field. It used max pooling operationfor reducing the size of image and making noise robust. However, instead ofreducing the complexity of the model, max pooling has the disadvantageof omitting some information about the image in reducing it. So, thispaper used two diagonal elements of down-sampling operation instead ofit. We think that the down-sampling feature maps have more informationintrinsically than max pooling feature maps because of keeping the Nyquisttheorem and extracting the latent information from them. In addition,this paper used two other diagonal elements for the skip connection. Indecoding, we used Subpixel Convolution rather than transposed convolutionto efficiently decode the encoded feature maps. Including all the ideas, thispaper proposed the new encoder-decoder model called Down-Sampling andSubpixel Convolution U-Net (DSSC-UNet). To prove the better performanceof the proposed model, this paper measured the performance of the UNetand DSSC-UNet on the Cityscapes. As a result, DSSC-UNet achieved89.6% Mean Intersection OverUnion (Mean-IoU) andU-Net achieved 85.6%Mean-IoU, confirming that DSSC-UNet achieved better performance.

A Survey on Image Semantic Segmentation Using Deep Learning Techniques

Image semantic segmentation is an important branch of computer vision of a wide variety of practical applications such as medical image analysis,autonomous driving,virtual or augmented reality,etc.In recent years,due to the remarkable performance of transformer and multilayer perceptron(MLP)in computer vision,which is equivalent to convolutional neural network(CNN),there has been a substantial amount of image semantic segmentation works aimed at developing different types of deep learning architecture.This survey aims to provide a comprehensive overview of deep learning methods in the field of general image semantic segmentation.Firstly,the commonly used image segmentation datasets are listed.Next,extensive pioneering works are deeply studied from multiple perspectives(e.g.,network structures,feature fusion methods,attention mechanisms),and are divided into four categories according to different network architectures:CNN-based architectures,transformer-based architectures,MLP-based architectures,and others.Furthermore,this paper presents some common evaluation metrics and compares the respective advantages and limitations of popular techniques both in terms of architectural design and their experimental value on the most widely used datasets.Finally,possible future research directions and challenges are discussed for the reference of other researchers.

Image Semantic Segmentation for Autonomous Driving Based on Improved U-Net

Image semantic segmentation has become an essential part of autonomous driving.To further improve the generalization ability and the robustness of semantic segmentation algorithms,a lightweight algorithm network based on Squeeze-and-Excitation Attention Mechanism(SE)and Depthwise Separable Convolution(DSC)is designed.Meanwhile,Adam-GC,an Adam optimization algorithm based on Gradient Compression(GC),is proposed to improve the training speed,segmentation accuracy,generalization ability and stability of the algorithm network.To verify and compare the effectiveness of the algorithm network proposed in this paper,the trained networkmodel is used for experimental verification and comparative test on the Cityscapes semantic segmentation dataset.The validation and comparison results show that the overall segmentation results of the algorithmnetwork can achieve 78.02%MIoU on Cityscapes validation set,which is better than the basic algorithm network and the other latest semantic segmentation algorithms network.Besides meeting the stability and accuracy requirements,it has a particular significance for the development of image semantic segmentation.

Adaptive Boundary and Semantic Composite Segmentation Method for Individual Objects in Aerial Images

There are two types of methods for image segmentation.One is traditional image processing methods,which are sensitive to details and boundaries,yet fail to recognize semantic information.The other is deep learning methods,which can locate and identify different objects,but boundary identifications are not accurate enough.Both of them cannot generate entire segmentation information.In order to obtain accurate edge detection and semantic information,an Adaptive Boundary and Semantic Composite Segmentation method(ABSCS)is proposed.This method can precisely semantic segment individual objects in large-size aerial images with limited GPU performances.It includes adaptively dividing and modifying the aerial images with the proposed principles and methods,using the deep learning method to semantic segment and preprocess the small divided pieces,using three traditional methods to segment and preprocess original-size aerial images,adaptively selecting traditional results tomodify the boundaries of individual objects in deep learning results,and combining the results of different objects.Individual object semantic segmentation experiments are conducted by using the AeroScapes dataset,and their results are analyzed qualitatively and quantitatively.The experimental results demonstrate that the proposed method can achieve more promising object boundaries than the original deep learning method.This work also demonstrates the advantages of the proposed method in applications of point cloud semantic segmentation and image inpainting.

CFSA-Net:Efficient Large-Scale Point Cloud Semantic Segmentation Based on Cross-Fusion Self-Attention

Traditional models for semantic segmentation in point clouds primarily focus on smaller scales.However,in real-world applications,point clouds often exhibit larger scales,leading to heavy computational and memory requirements.The key to handling large-scale point clouds lies in leveraging random sampling,which offers higher computational efficiency and lower memory consumption compared to other sampling methods.Nevertheless,the use of random sampling can potentially result in the loss of crucial points during the encoding stage.To address these issues,this paper proposes cross-fusion self-attention network(CFSA-Net),a lightweight and efficient network architecture specifically designed for directly processing large-scale point clouds.At the core of this network is the incorporation of random sampling alongside a local feature extraction module based on cross-fusion self-attention(CFSA).This module effectively integrates long-range contextual dependencies between points by employing hierarchical position encoding(HPC).Furthermore,it enhances the interaction between each point’s coordinates and feature information through cross-fusion self-attention pooling,enabling the acquisition of more comprehensive geometric information.Finally,a residual optimization(RO)structure is introduced to extend the receptive field of individual points by stacking hierarchical position encoding and cross-fusion self-attention pooling,thereby reducing the impact of information loss caused by random sampling.Experimental results on the Stanford Large-Scale 3D Indoor Spaces(S3DIS),Semantic3D,and SemanticKITTI datasets demonstrate the superiority of this algorithm over advanced approaches such as RandLA-Net and KPConv.These findings underscore the excellent performance of CFSA-Net in large-scale 3D semantic segmentation.

A Lightweight Road Scene Semantic Segmentation Algorithm

In recent years,with the continuous deepening of smart city construction,there have been significant changes and improvements in the field of intelligent transportation.The semantic segmentation of road scenes has important practical significance in the fields of automatic driving,transportation planning,and intelligent transportation systems.However,the current mainstream lightweight semantic segmentation models in road scene segmentation face problems such as poor segmentation performance of small targets and insufficient refinement of segmentation edges.Therefore,this article proposes a lightweight semantic segmentation model based on the LiteSeg model improvement to address these issues.The model uses the lightweight backbone network MobileNet instead of the LiteSeg backbone network to reduce the network parameters and computation,and combines the Coordinate Attention(CA)mechanism to help the network capture long-distance dependencies.At the same time,by combining the dependencies of spatial information and channel information,the Spatial and Channel Network(SCNet)attention mechanism is proposed to improve the feature extraction ability of the model.Finally,a multiscale transposed attention encoding(MTAE)module was proposed to obtain features of different resolutions and perform feature fusion.In this paper,the proposed model is verified on the Cityscapes dataset.The experimental results show that the addition of SCNet and MTAE modules increases the mean Intersection over Union(mIoU)of the original LiteSeg model by 4.69%.On this basis,the backbone network is replaced with MobileNet,and the CA model is added at the same time.At the cost of increasing the minimum model parameters and computing costs,the mIoU of the original LiteSeg model is increased by 2.46%.This article also compares the proposed model with some current lightweight semantic segmentation models,and experiments show that the comprehensive performance of the proposed model is the best,especially in achieving excellent results in small object segmentation.Finally,this article will conduct generalization testing on the KITTI dataset for the proposed model,and the experimental results show that the proposed algorithm has a certain degree of generalization.

3D Point Cloud Semantic Segmentation Based PAConv and SE_variant

With the increasing popularity of 3D sensors(e.g.,Kinect)and light field cameras,technologies such as driverless,smart home and virtual reality have become hot spots for engineering applications.As an important part of 3D vision tasks,point cloud semantic segmentation has received a lot of attention from researchers.In this work,we focus on realistically collected indoor point cloud data and propose a point cloud semantic segmentation method based on PAConv and SE_variant.The SE_variant module captures global perception from a broad perspective of feature space by fusing different pooling methods,which fully utilize the channel information of point clouds.The effectiveness of the method is verified by comparing with other methods on S3DIS and ScanNetV2 semantic tagging benchmarks,and achieving 65.3%mIoU in S3DIS,47.6%mIoU in ScanNetV2.The results of the ablation experiments verify the effectiveness of the key modules and analyze how to use the attention mechanism to improve the 3D semantic segmentation performance.

Enhancing Semantic Segmentation through Reinforced Active Learning: Combating Dataset Imbalances and Bolstering Annotation Efficiency

This research addresses the challenges of training large semantic segmentation models for image analysis,focusing on expediting the annotation process and mitigating imbalanced datasets.In the context of imbalanced datasets,biases related to age and gender in clinical contexts and skewed representation in natural images can affect model performance.Strategies to mitigate these biases are explored to enhance efficiency and accuracy in semantic segmentation analysis.An in-depth exploration of various reinforced active learning methodologies for image segmentation is conducted,optimizing precision and efficiency across diverse domains.The proposed framework integrates Dueling Deep Q-Networks(DQN),Prioritized Experience Replay,Noisy Networks,and Emphasizing Recent Experience.Extensive experimentation and evaluation of diverse datasets reveal both improvements and limitations associated with various approaches in terms of overall accuracy and efficiency.This research contributes to the expansion of reinforced active learning methodologies for image segmentation,paving the way for more sophisticated and precise segmentation algorithms across diverse domains.The findings emphasize the need for a careful balance between exploration and exploitation strategies in reinforcement learning for effective image segmentation.

Enhanced Wolf Pack Algorithm (EWPA) and Dense-kUNet Segmentation for Arterial Calcifications in Mammograms

Breast Arterial Calcification(BAC)is a mammographic decision dissimilar to cancer and commonly observed in elderly women.Thus identifying BAC could provide an expense,and be inaccurate.Recently Deep Learning(DL)methods have been introduced for automatic BAC detection and quantification with increased accuracy.Previously,classification with deep learning had reached higher efficiency,but designing the structure of DL proved to be an extremely challenging task due to overfitting models.It also is not able to capture the patterns and irregularities presented in the images.To solve the overfitting problem,an optimal feature set has been formed by Enhanced Wolf Pack Algorithm(EWPA),and their irregularities are identified by Dense-kUNet segmentation.In this paper,Dense-kUNet for segmentation and optimal feature has been introduced for classification(severe,mild,light)that integrates DenseUNet and kU-Net.Longer bound links exist among adjacent modules,allowing relatively rough data to be sent to the following component and assisting the system in finding higher qualities.The major contribution of the work is to design the best features selected by Enhanced Wolf Pack Algorithm(EWPA),and Modified Support Vector Machine(MSVM)based learning for classification.k-Dense-UNet is introduced which combines the procedure of Dense-UNet and kU-Net for image segmentation.Longer bound associations occur among nearby sections,allowing relatively granular data to be sent to the next subsystem and benefiting the system in recognizing smaller characteristics.The proposed techniques and the performance are tested using several types of analysis techniques 826 filled digitized mammography.The proposed method achieved the highest precision,recall,F-measure,and accuracy of 84.4333%,84.5333%,84.4833%,and 86.8667%when compared to other methods on the Digital Database for Screening Mammography(DDSM).