Real-Time Detection and Instance Segmentation of Strawberry in Unstructured Environment

The real-time detection and instance segmentation of strawberries constitute fundamental components in the development of strawberry harvesting robots.Real-time identification of strawberries in an unstructured envi-ronment is a challenging task.Current instance segmentation algorithms for strawberries suffer from issues such as poor real-time performance and low accuracy.To this end,the present study proposes an Efficient YOLACT(E-YOLACT)algorithm for strawberry detection and segmentation based on the YOLACT framework.The key enhancements of the E-YOLACT encompass the development of a lightweight attention mechanism,pyramid squeeze shuffle attention(PSSA),for efficient feature extraction.Additionally,an attention-guided context-feature pyramid network(AC-FPN)is employed instead of FPN to optimize the architecture’s performance.Furthermore,a feature-enhanced model(FEM)is introduced to enhance the prediction head’s capabilities,while efficient fast non-maximum suppression(EF-NMS)is devised to improve non-maximum suppression.The experimental results demonstrate that the E-YOLACT achieves a Box-mAP and Mask-mAP of 77.9 and 76.6,respectively,on the custom dataset.Moreover,it exhibits an impressive category accuracy of 93.5%.Notably,the E-YOLACT also demonstrates a remarkable real-time detection capability with a speed of 34.8 FPS.The method proposed in this article presents an efficient approach for the vision system of a strawberry-picking robot.

Multi-Scale Mixed Attention Tea Shoot Instance Segmentation Model

Tea leaf picking is a crucial stage in tea production that directly influences the quality and value of the tea.Traditional tea-picking machines may compromise the quality of the tea leaves.High-quality teas are often handpicked and need more delicate operations in intelligent picking machines.Compared with traditional image processing techniques,deep learning models have stronger feature extraction capabilities,and better generalization and are more suitable for practical tea shoot harvesting.However,current research mostly focuses on shoot detection and cannot directly accomplish end-to-end shoot segmentation tasks.We propose a tea shoot instance segmentation model based on multi-scale mixed attention(Mask2FusionNet)using a dataset from the tea garden in Hangzhou.We further analyzed the characteristics of the tea shoot dataset,where the proportion of small to medium-sized targets is 89.9%.Our algorithm is compared with several mainstream object segmentation algorithms,and the results demonstrate that our model achieves an accuracy of 82%in recognizing the tea shoots,showing a better performance compared to other models.Through ablation experiments,we found that ResNet50,PointRend strategy,and the Feature Pyramid Network(FPN)architecture can improve performance by 1.6%,1.4%,and 2.4%,respectively.These experiments demonstrated that our proposed multi-scale and point selection strategy optimizes the feature extraction capability for overlapping small targets.The results indicate that the proposed Mask2FusionNet model can perform the shoot segmentation in unstructured environments,realizing the individual distinction of tea shoots,and complete extraction of the shoot edge contours with a segmentation accuracy of 82.0%.The research results can provide algorithmic support for the segmentation and intelligent harvesting of premium tea shoots at different scales.

Dynamic SLAM Visual Odometry Based on Instance Segmentation:A Comprehensive Review

Dynamic Simultaneous Localization and Mapping(SLAM)in visual scenes is currently a major research area in fields such as robot navigation and autonomous driving.However,in the face of complex real-world envi-ronments,current dynamic SLAM systems struggle to achieve precise localization and map construction.With the advancement of deep learning,there has been increasing interest in the development of deep learning-based dynamic SLAM visual odometry in recent years,and more researchers are turning to deep learning techniques to address the challenges of dynamic SLAM.Compared to dynamic SLAM systems based on deep learning methods such as object detection and semantic segmentation,dynamic SLAM systems based on instance segmentation can not only detect dynamic objects in the scene but also distinguish different instances of the same type of object,thereby reducing the impact of dynamic objects on the SLAM system’s positioning.This article not only introduces traditional dynamic SLAM systems based on mathematical models but also provides a comprehensive analysis of existing instance segmentation algorithms and dynamic SLAM systems based on instance segmentation,comparing and summarizing their advantages and disadvantages.Through comparisons on datasets,it is found that instance segmentation-based methods have significant advantages in accuracy and robustness in dynamic environments.However,the real-time performance of instance segmentation algorithms hinders the widespread application of dynamic SLAM systems.In recent years,the rapid development of single-stage instance segmentationmethods has brought hope for the widespread application of dynamic SLAM systems based on instance segmentation.Finally,possible future research directions and improvementmeasures are discussed for reference by relevant professionals.

Coarse-to-Fine Video Instance Segmentation With Factorized Conditional Appearance Flows

We introduce a novel method using a new generative model that automatically learns effective representations of the target and background appearance to detect,segment and track each instance in a video sequence.Differently from current discriminative tracking-by-detection solutions,our proposed hierarchical structural embedding learning can predict more highquality masks with accurate boundary details over spatio-temporal space via the normalizing flows.We formulate the instance inference procedure as a hierarchical spatio-temporal embedded learning across time and space.Given the video clip,our method first coarsely locates pixels belonging to a particular instance with Gaussian distribution and then builds a novel mixing distribution to promote the instance boundary by fusing hierarchical appearance embedding information in a coarse-to-fine manner.For the mixing distribution,we utilize a factorization condition normalized flow fashion to estimate the distribution parameters to improve the segmentation performance.Comprehensive qualitative,quantitative,and ablation experiments are performed on three representative video instance segmentation benchmarks(i.e.,YouTube-VIS19,YouTube-VIS21,and OVIS)and the effectiveness of the proposed method is demonstrated.More impressively,the superior performance of our model on an unsupervised video object segmentation dataset(i.e.,DAVIS19)proves its generalizability.Our algorithm implementations are publicly available at https://github.com/zyqin19/HEVis.

FIR-YOLACT:Fusion of ICIoU and Res2Net for YOLACT on Real-Time Vehicle Instance Segmentation

Autonomous driving technology has made a lot of outstanding achievements with deep learning,and the vehicle detection and classification algorithm has become one of the critical technologies of autonomous driving systems.The vehicle instance segmentation can perform instance-level semantic parsing of vehicle information,which is more accurate and reliable than object detection.However,the existing instance segmentation algorithms still have the problems of poor mask prediction accuracy and low detection speed.Therefore,this paper proposes an advanced real-time instance segmentation model named FIR-YOLACT,which fuses the ICIoU(Improved Complete Intersection over Union)and Res2Net for the YOLACT algorithm.Specifically,the ICIoU function can effectively solve the degradation problem of the original CIoU loss function,and improve the training convergence speed and detection accuracy.The Res2Net module fused with the ECA(Efficient Channel Attention)Net is added to the model’s backbone network,which improves the multi-scale detection capability and mask prediction accuracy.Furthermore,the Cluster NMS(Non-Maximum Suppression)algorithm is introduced in the model’s bounding box regression to enhance the performance of detecting similarly occluded objects.The experimental results demonstrate the superiority of FIR-YOLACT to the based methods and the effectiveness of all components.The processing speed reaches 28 FPS,which meets the demands of real-time vehicle instance segmentation.

SPM-IS: An auto-algorithm to acquire a mature soybean phenotype based on instance segmentation

Mature soybean phenotyping is an important process in soybean breeding;however, the manual process is time-consuming and labor-intensive. Therefore, a novel approach that is rapid, accurate and highly precise is required to obtain the phenotypic data of soybean stems, pods and seeds. In this research, we propose a mature soybean phenotype measurement algorithm called Soybean Phenotype Measure-instance Segmentation(SPM-IS). SPM-IS is based on a feature pyramid network, Principal Component Analysis(PCA) and instance segmentation. We also propose a new method that uses PCA to locate and measure the length and width of a target object via image instance segmentation. After 60,000 iterations, the maximum mean Average Precision(m AP) of the mask and box was able to reach 95.7%. The correlation coefficients R^(2) of the manual measurement and SPM-IS measurement of the pod length, pod width, stem length, complete main stem length, seed length and seed width were 0.9755, 0.9872, 0.9692, 0.9803,0.9656, and 0.9716, respectively. The correlation coefficients R^(2) of the manual counting and SPM-IS counting of pods, stems and seeds were 0.9733, 0.9872, and 0.9851, respectively. The above results show that SPM-IS is a robust measurement and counting algorithm that can reduce labor intensity, improve efficiency and speed up the soybean breeding process.

3D Instance Segmentation Using Deep Learning on RGB-D Indoor Data

3D object recognition is a challenging task for intelligent and robot systems in industrial and home indoor environments.It is critical for such systems to recognize and segment the 3D object instances that they encounter on a frequent basis.The computer vision,graphics,and machine learning fields have all given it a lot of attention.Traditionally,3D segmentation was done with hand-crafted features and designed approaches that didn’t achieve acceptable performance and couldn’t be generalized to large-scale data.Deep learning approaches have lately become the preferred method for 3D segmentation challenges by their great success in 2D computer vision.However,the task of instance segmentation is currently less explored.In this paper,we propose a novel approach for efficient 3D instance segmentation using red green blue and depth(RGB-D)data based on deep learning.The 2D region based convolutional neural networks(Mask R-CNN)deep learning model with point based rending module is adapted to integrate with depth information to recognize and segment 3D instances of objects.In order to generate 3D point cloud coordinates(x,y,z),segmented 2D pixels(u,v)of recognized object regions in the RGB image are merged into(u,v)points of the depth image.Moreover,we conducted an experiment and analysis to compare our proposed method from various points of view and distances.The experimentation shows the proposed 3D object recognition and instance segmentation are sufficiently beneficial to support object handling in robotic and intelligent systems.

C1M2:a universal algorithm for 3D instance segmentation,annotation,and quantification of irregular cells

Cell instance segmentation is a fundamental task for many biological applications,especially for packed cells in three-dimensional(3D)microscope images that can fully display cellular morphology.Image processing algorithms based on neural networks and feature engineering have enabled great progress in two-dimensional(2D)instance segmentation.However,current methods cannot achieve high segmentation accuracy for irregular cells in 3D images.In this study,we introduce a universal,morphology-based 3D instance segmentation algorithm called Crop Once Merge Twice(C1M2),which can segment cells from a wide range of image types and does not require nucleus images.C1M2 can be extended to quantify the fluorescence intensity of fluorescent proteins and antibodies and automatically annotate their expression levels in individual cells.Our results suggest that C1M2 can serve as a tissue cytometry for 3D histopathological assays by quantifying fluorescence intensity with spatial localization and morphological information.

YOLO-CORE: Contour Regression for Efficient Instance Segmentation

Instance segmentation has drawn mounting attention due to its significant utility.However,high computational costs have been widely acknowledged in this domain,as the instance mask is generally achieved by pixel-level labeling.In this paper,we present a conceptually efficient contour regression network based on the you only look once(YOLO)architecture named YOLO-CORE for instance segmentation.The mask of the instance is efficiently acquired by explicit and direct contour regression using our designed multiorder constraint consisting of a polar distance loss and a sector loss.Our proposed YOLO-CORE yields impressive segmentation performance in terms of both accuracy and speed.It achieves 57.9%AP@0.5 with 47 FPS(frames per second)on the semantic boundaries dataset(SBD)and 51.1%AP@0.5 with 46 FPS on the COCO dataset.The superior performance achieved by our method with explicit contour regression suggests a new technique line in the YOLO-based image understanding field.Moreover,our instance segmentation design can be flexibly integrated into existing deep detectors with negligible computation cost(65.86 BFLOPs(billion float operations per second)to 66.15 BFLOPs with the YOLOv3 detector).

Detection method for the cucumber robotic grasping pose in clutter scenarios via instance segmentation

The application of robotic grasping for agricultural products pushes automation in agriculture-related industries.Cucumber,a common vegetable in greenhouses and supermarkets,often needs to be grasped from a cluttered scene.In order to realize efficient grasping in cluttered scenes,a fully automatic cucumber recognition,grasping,and palletizing robot system was constructed in this paper.The system adopted Yolact++deep learning network to segment cucumber instances.An early fusion method of F-RGBD was proposed,which increases the algorithm's discriminative ability for these appearance-similar cucumbers at different depths,and at different occlusion degrees.The results of the comparative experiment of the F-RGBD dataset and the common RGB dataset on Yolact++prove the positive effect of the F-RGBD fusion method.Its segmentation masks have higher quality,are more continuous,and are less false positive for prioritizing-grasping prediction.Based on the segmentation result,a 4D grab line prediction method was proposed for cucumber grasping.And the cucumber detection experiment in cluttered scenarios is carried out in the real world.The success rate is 93.67%and the average sorting time is 9.87 s.The effectiveness of the cucumber segmentation and grasping pose acquisition method is verified by experiments.

Semantic segmentation-assisted instance feature fusion for multi-level 3D part instance segmentation

Recognizing 3D part instances from a 3D point cloud is crucial for 3D structure and scene understanding.Several learning-based approaches use semantic segmentation and instance center prediction as training tasks and fail to further exploit the inherent relationship between shape semantics and part instances.In this paper,we present a new method for 3D part instance segmentation.Our method exploits semantic segmentation to fuse nonlocal instance features,such as center prediction,and further enhances the fusion scheme in a multi-and cross-level way.We also propose a semantic region center prediction task to train and leverage the prediction results to improve the clustering of instance points.Our method outperforms existing methods with a large-margin improvement in the PartNet benchmark.We also demonstrate that our feature fusion scheme can be applied to other existing methods to improve their performance in indoor scene instance segmentation tasks.

Improved Convolutional Neural Network for Traffic Scene Segmentation

In actual traffic scenarios,precise recognition of traffic participants,such as vehicles and pedestrians,is crucial for intelligent transportation.This study proposes an improved algorithm built on Mask-RCNN to enhance the ability of autonomous driving systems to recognize traffic participants.The algorithmincorporates long and shortterm memory networks and the fused attention module(GSAM,GCT,and Spatial Attention Module)to enhance the algorithm’s capability to process both global and local information.Additionally,to increase the network’s initial operation stability,the original network activation function was replaced with Gaussian error linear unit.Experiments were conducted using the publicly available Cityscapes dataset.Comparing the test results,it was observed that the revised algorithmoutperformed the original algorithmin terms of AP_(50),AP_(75),and othermetrics by 8.7%and 9.6%for target detection and 12.5%and 13.3%for segmentation.

Visual inspection of aircraft skin:Automated pixel-level defect detection by instance segmentation

Skin defect inspection is one of the most significant tasks in the conventional process of aircraft inspection.This paper proposes a vision-based method of pixel-level defect detection,which is based on the Mask Scoring R-CNN.First,an attention mechanism and a feature fusion module are introduced,to improve feature representation.Second,a new classifier head—consisting of four convolutional layers and a fully connected layer—is proposed,to reduce the influence of information around the area of the defect.Third,to evaluate the proposed method,a dataset of aircraft skin defects was constructed,containing 276 images with a resolution of 960×720 pixels.Experimental results show that the proposed classifier head improves the detection and segmentation accuracy,for aircraft skin defect inspection,more effectively than the attention mechanism and feature fusion module.Compared with the Mask R-CNN and Mask Scoring R-CNN,the proposed method increased the segmentation precision by approximately 21%and 19.59%,respectively.These results demonstrate that the proposed method performs favorably against the other two methods of pixellevel aircraft skin defect detection.

Deep Learning-Based 3D Instance and Semantic Segmentation: A Review

The process of segmenting point cloud data into several homogeneous areas with points in the same region having the same attributes is known as 3D segmentation.Segmentation is challenging with point cloud data due to substantial redundancy,fluctuating sample density and lack of apparent organization.The research area has a wide range of robotics applications,including intelligent vehicles,autonomous mapping and navigation.A number of researchers have introduced various methodologies and algorithms.Deep learning has been successfully used to a spectrum of 2D vision domains as a prevailing A.I.methods.However,due to the specific problems of processing point clouds with deep neural networks,deep learning on point clouds is still in its initial stages.This study examines many strategies that have been presented to 3D instance and semantic segmentation and gives a complete assessment of current developments in deep learning-based 3D segmentation.In these approaches’benefits,draw backs,and design mechanisms are studied and addressed.This study evaluates the impact of various segmentation algorithms on competitiveness on various publicly accessible datasets,as well as the most often used pipelines,their advantages and limits,insightful findings and intriguing future research directions.

Research on the Preservation Method of Traditional Village Roof Information:A Case Study of Gubeikou Village

Traditional Chinese villages serve as crucial repositories of traditional culture.However,In China,the urgent task of preserving information about traditional village architecture has arisen due to the degradation of these villages’appearance caused by rapid urbanization in recent years.This paper proposes a method for preserving information about traditional village rooftops based on high spatial resolution remote sensing imagery.Leveraging an improved Mask R-CNN model,the method conducts target recognition on the rooftops of traditional village buildings and generates vectorized representations of these rooftops.The precision rate,recall rate,and F1-score achieved in the experimental results are 93.26%,86.33%,and 92.02%,respectively.These findings indicate the effectiveness of the proposed method in preserving information about traditional village architecture and providing a viable approach to support the sustainable development of traditional villages in China.

High-Resolution Remote Sensing Imagery for the Recognition of Traditional Villages

Traditional Chinese villages,vital carriers of traditional culture,have faced significant alterations due to urbanization in recent years,urgently necessitating artificial intelligence data updates.This study integrates high spatial resolution remote sensing imagery with deep learning techniques,proposing a novel method for identifying rooftops of traditional Chinese village buildings using high-definition remote sensing images.Using 0.54 m spatial resolution imagery of traditional village areas as the data source,this method analyzes the geometric and spectral image characteristics of village building rooftops.It constructs a deep learning feature sample library tailored to the target types.Employing a semantically enhanced version of the improved Mask R-CNN(Mask Region-based Convolutional Neural Network)for building recognition,the study conducts experiments on localized imagery from different regions.The results demonstrated that the modified Mask R-CNN effectively identifies traditional village building rooftops,achieving an of 0.7520 and an of 0.7400.It improves the current problem of misidentification and missed detection caused by feature heterogeneity.This method offers a viable and effective approach for industrialized data monitoring of traditional villages,contributing to their sustainable development.

A Lane Detection Method Based on Semantic Segmentation

This paper proposes a novel method of lane detection,which adopts VGG16 as the basis of convolutional neural network to extract lane line features by cavity convolution,wherein the lane lines are divided into dotted lines and solid lines.Expanding the field of experience through hollow convolution,the full connection layer of the network is discarded,the last largest pooling layer of the VGG16 network is removed,and the processing of the last three convolution layers is replaced by hole convolution.At the same time,CNN adopts the encoder and decoder structure mode,and uses the index function of the maximum pooling layer in the decoder part to upsample the encoder in a counter-pooling manner,realizing semantic segmentation.And combined with the instance segmentation,and finally through the fitting to achieve the detection of the lane line.In addition,the currently disclosed lane line data sets are relatively small,and there is no distinction between lane solid lines and dashed lines.To this end,our work made a lane line data set for the lane virtual and real identification,and based on the proposed algorithm effective verification of the data set achieved by the increased segmentation.The final test shows that the proposed method has a good balance between lane detection speed and accuracy,which has good robustness.

High-Movement Human Segmentation in Video Using Adaptive N-Frames Ensemble

Awide range of camera apps and online video conferencing services support the feature of changing the background in real-time for aesthetic,privacy,and security reasons.Numerous studies show that theDeep-Learning(DL)is a suitable option for human segmentation,and the ensemble of multiple DL-based segmentation models can improve the segmentation result.However,these approaches are not as effective when directly applied to the image segmentation in a video.This paper proposes an Adaptive N-Frames Ensemble(AFE)approach for high-movement human segmentation in a video using an ensemble of multiple DL models.In contrast to an ensemble,which executes multiple DL models simultaneously for every single video frame,the proposed AFE approach executes only a single DL model upon a current video frame.It combines the segmentation outputs of previous frames for the final segmentation output when the frame difference is less than a particular threshold.Our method employs the idea of the N-Frames Ensemble(NFE)method,which uses the ensemble of the image segmentation of a current video frame and previous video frames.However,NFE is not suitable for the segmentation of fast-moving objects in a video nor a video with low frame rates.The proposed AFE approach addresses the limitations of the NFE method.Our experiment uses three human segmentation models,namely Fully Convolutional Network(FCN),DeepLabv3,and Mediapipe.We evaluated our approach using 1711 videos of the TikTok50f dataset with a single-person view.The TikTok50f dataset is a reconstructed version of the publicly available TikTok dataset by cropping,resizing and dividing it into videos having 50 frames each.This paper compares the proposed AFE with single models and the Two-Models Ensemble,as well as the NFE models.The experiment results show that the proposed AFE is suitable for low-movement as well as high-movement human segmentation in a video.

A Fast Panoptic Segmentation Network for Self-Driving Scene Understanding

In recent years,a gain in popularity and significance of science understanding has been observed due to the high paced progress in computer vision techniques and technologies.The primary focus of computer vision based scene understanding is to label each and every pixel in an image as the category of the object it belongs to.So it is required to combine segmentation and detection in a single framework.Recently many successful computer vision methods has been developed to aid scene understanding for a variety of real world application.Scene understanding systems typically involves detection and segmentation of different natural and manmade things.A lot of research has been performed in recent years,mostly with a focus on things(a well-defined objects that has shape,orientations and size)with a less focus on stuff classes(amorphous regions that are unclear and lack a shape,size or other characteristics Stuff region describes many aspects of scene,like type,situation,environment of scene etc.and hence can be very helpful in scene understanding.Existing methods for scene understanding still have to cover a challenging path to cope up with the challenges of computational time,accuracy and robustness for varying level of scene complexity.A robust scene understanding method has to effectively deal with imbalanced distribution of classes,overlapping objects,fuzzy object boundaries and poorly localized objects.The proposed method presents Panoptic Segmentation on Cityscapes Dataset.Mobilenet-V2 is used as a backbone for feature extraction that is pre-trained on ImageNet.MobileNet-V2 with state-of-art encoder-decoder architecture of DeepLabV3+with some customization and optimization is employed Atrous convolution along with Spatial Pyramid Pooling are also utilized in the proposed method to make it more accurate and robust.Very promising and encouraging results have been achieved that indicates the potential of the proposed method for robust scene understanding in a fast and reliable way.

Research on Automatic Elimination of Laptop Computer in Security CT Images Based on Projection Algorithm and YOLOv7-Seg

In civil aviation security screening, laptops, with their intricate structural composition, provide the potential for criminals to conceal dangerous items. Presently, the security process necessitates passengers to individually present their laptops for inspection. The paper introduced a method for laptop removal. By combining projection algorithms with the YOLOv7-Seg model, a laptop’s three views were generated through projection, and instance segmentation of these views was achieved using YOLOv7-Seg. The resulting 2D masks from instance segmentation at different angles were employed to reconstruct a 3D mask through angle restoration. Ultimately, the intersection of this 3D mask with the original 3D data enabled the successful extraction of the laptop’s 3D information. Experimental results demonstrated that the fusion of projection and instance segmentation facilitated the automatic removal of laptops from CT data. Moreover, higher instance segmentation model accuracy leads to more precise removal outcomes. By implementing the laptop removal functionality, the civil aviation security screening process becomes more efficient and convenient. Passengers will no longer be required to individually handle their laptops, effectively enhancing the efficiency and accuracy of security screening.