ASRNet: Adversarial Segmentation and Registration Networks for Multispectral Fundus Images

Multispectral imaging (MSI) technique is often used to capture imagesof the fundus by illuminating it with different wavelengths of light. However,these images are taken at different points in time such that eyeball movementscan cause misalignment between consecutive images. The multispectral imagesequence reveals important information in the form of retinal and choroidal bloodvessel maps, which can help ophthalmologists to analyze the morphology of theseblood vessels in detail. This in turn can lead to a high diagnostic accuracy of several diseases. In this paper, we propose a novel semi-supervised end-to-end deeplearning framework called “Adversarial Segmentation and Registration Nets”(ASRNet) for the simultaneous estimation of the blood vessel segmentation andthe registration of multispectral images via an adversarial learning process. ASRNet consists of two subnetworks: (i) A segmentation module S that fulfills theblood vessel segmentation task, and (ii) A registration module R that estimatesthe spatial correspondence of an image pair. Based on the segmention-drivenregistration network, we train the segmentation network using a semi-supervisedadversarial learning strategy. Our experimental results show that the proposedASRNet can achieve state-of-the-art accuracy in segmentation and registrationtasks performed with real MSI datasets.

Joint Deep Matching Model of OCT Retinal Layer Segmentation

Optical Coherence Tomography(OCT)is very important in medicine and provide useful diagnostic information.Measuring retinal layer thicknesses plays a vital role in pathophysiologic factors of many ocular conditions.Among the existing retinal layer segmentation approaches,learning or deep learning-based methods belong to the state-of-art.However,most of these techniques rely on manual-marked layers and the performances are limited due to the image quality.In order to overcome this limitation,we build a framework based on gray value curve matching,which uses depth learning to match the curve for semi-automatic segmentation of retinal layers from OCT.The depth convolution network learns the column correspondence in the OCT image unsupervised.The whole OCT image participates in the depth convolution neural network operation,compares the gray value of each column,and matches the gray value sequence of the transformation column and the next column.Using this algorithm,when a boundary point is manually specified,we can accurately segment the boundary between retinal layers.Our experimental results obtained from a 54-subjects database of both normal healthy eyes and affected eyes demonstrate the superior performances of our approach.

FCOS-LSC:A Novel Model for Green Fruit Detection in a Complex Orchard Environment

To better address the difficulties in designing green fruit recognition techniques in machine vision systems,a new fruit detection model is proposed.This model is an optimization of the FCOS(full convolution one-stage object detection)algorithm,incorporating LSC(level scales,spaces,channels)attention blocks in the network structure,and named FCOS-LSC.The method achieves efficient recognition and localization of green fruit images affected by overlapping occlusions,lighting conditions,and capture angles.Specifically,the improved feature extraction network ResNet50 with added deformable convolution is used to fully extract green fruit feature information.The feature pyramid network(FPN)is employed to fully fuse low-level detail information and high-level semantic information in a cross-connected and top-down connected way.Next,the attention mechanisms are added to each of the 3 dimensions of scale,space(including the height and width of the feature map),and channel of the generated multiscale feature map to improve the feature perception capability of the network.Finally,the classification and regression subnetworks of the model are applied to predict the fruit category and bounding box.In the classification branch,a new positive and negative sample selection strategy is applied to better distinguish supervised signals by designing weights in the loss function to achieve more accurate fruit detection.The proposed FCOS-LSC model has 38.65M parameters,38.72G floating point operations,and mean average precision of 63.0%and 75.2%for detecting green apples and green persimmons,respectively.In summary,FCOS-LSC outperforms the state-of-the-art models in terms of precision and complexity to meet the accurate and efficient requirements of green fruit recognition using intelligent agricultural equipment.Correspondingly,FCOS-LSC can be used to improve the robustness and generalization of the green fruit detection models.

SE-COTR:A Novel Fruit Segmentation Model for Green Apples Application in Complex Orchard

Because of the unstructured characteristics of natural orchards,the efficient detection and segmentation applications of green fruits remain an essential challenge for intelligent agriculture.Therefore,an innovative fruit segmentation method based on deep learning,termed SE-COTR(segmentation based on coordinate transformer),is proposed to achieve accurate and real-time segmentation of green apples.The lightweight network MobileNetV2 is used as the backbone,combined with the constructed coordinate attention-based coordinate transformer module to enhance the focus on effective features.In addition,joint pyramid upsampling module is optimized for integrating multiscale features,making the model suitable for the detection and segmentation of target fruits with different sizes.Finally,in combination with the outputs of the function heads,the dynamic convolution operation is applied to predict the instance mask.In complex orchard environment with variable conditions,SE-COTR achieves a mean average precision of 61.6%with low complexity for green apple fruit segmentation at severe occlusion and different fruit scales.Especially,the segmentation accuracy for small target fruits reaches 43.3%,which is obviously better than other advanced segmentation models and realizes good recognition results.The proposed method effectively solves the problem of low accuracy and overly complex fruit segmentation models with the same color as the background and can be built in portable mobile devices to undertake accurate and efficient agricultural works in complex orchard.

Preprocessing method of night vision image application in apple harvesting robot

Due to the low working efficiency of apple harvesting robots,there is still a long way to go for commercialization.The machine performance and extended operating time are the two research aspects for improving efficiencies of harvesting robots,this study focused on the extended operating time and proposed a round-the-clock operation mode.Due to the influences of light,temperature,humidity,etc.,the working environment at night is relatively complex,and thus restricts the operating efficiency of the apple harvesting robot.Three different artificial light sources(incandescent lamp,fluorescent lamp,and LED lights)were selected for auxiliary light according to certain rules so that the apple night vision images could be captured.In addition,by color analysis,night and natural light images were compared to find out the color characteristics of the night vision images,and intuitive visual and difference image methods were used to analyze the noise characteristics.The results showed that the incandescent lamp is the best artificial auxiliary light for apple harvesting robots working at night,and the type of noise contained in apple night vision images is Gaussian noise mixed with some salt and pepper noise.The preprocessing method can provide a theoretical and technical reference for subsequent image processing.

BFP Net: Balanced Feature Pyramid Network for Small Apple Detection in Complex Orchard Environment

Despite of significant achievements made in the detection of target fruits,small fruit detection remains a great challenge,especially for immature small green fruits with a few pixels.The closeness of color between the fruit skin and the background greatly increases the difficulty of locating small target fruits in the natural orchard environment.In this paper,we propose a balanced feature pyramid network(BFP Net)for small apple detection.

SE-COTR: A Novel Fruit Segmentation Model for Green Apples Application in Complex Orchard

Because of the unstructured characteristics of natural orchards,the efficient detection and segmentation applications of green fruits remain an essential challenge for intelligent agriculture.Therefore,an innovative fruit segmentation method based on deep learning,termed SE-COTR(segmentation based on coordinate transformer),is proposed to achieve accurate and real-time segmentation of green apples.

Novel green-fruit detection algorithm based on D2D framework

In the complex orchard environment,the efficient and accurate detection of object fruit is the basic requirement to realize the orchard yield measurement and automatic harvesting.Sometimes it is hard to differentiate between the object fruits and the background because of the similar color,and it is challenging due to the ambient light and camera angle by which the photos have been taken.These problems make it hard to detect green fruits in orchard environments.In this study,a two-stage dense to detection framework(D2D)was proposed to detect green fruits in orchard environments.The proposed model was based on multi-scale feature extraction of target fruit by using feature pyramid networks MobileNetV2+FPN structure and generated region proposal of target fruit by using Region Proposal Network(RPN)structure.In the regression branch,the offset of each local feature was calculated,and the positive and negative samples of the region proposals were predicted by a binary mask prediction to reduce the interference of the background to the prediction box.In the classification branch,features were extracted from each sub-region of the region proposal,and features with distinguishing information were obtained through adaptive weighted pooling to achieve accurate classification.The new proposed model adopted an anchor-free frame design,which improves the generalization ability,makes the model more robust,and reduces the storage requirements.The experimental results of persimmon and green apple datasets show that the new model has the best detection performance,which can provide theoretical reference for other green object detection.