Detection of abnormal hydroponic lettuce leaves based on image processing and machine learning

Accurate and fast detection of abnormal hydroponic lettuce leaves is primary technology for robotic sorting.Yellow and rotten leaves are main types of abnormal leaves in hydroponic lettuce.This study aims to demonstrate a feasibility of detecting yellow and rotten leaves of hydroponic lettuce by machine learning models,i.e.Multiple Linear Regression(MLR),K-Nearest Neighbor(KNN),and Support Vector Machine(SVM).One-way analysis of variance was applied to reduce RGB,HSV,and L*a*b*features number of hydroponic lettuce images.Image binarization,image mask,and image filling methods were employed to segment hydroponic lettuce from an image for models testing.Results showed that G,H,and a*were selected from RGB,HSV,and L*a*b*for training models.It took about 20.25 s to detect an image with 3024
4032 pixels by KNN,which was much longer than MLR(0.61 s)and SVM(1.98 s).MLR got detection accuracies of 89.48%and 99.29%for yellow and rotten leaves,respectively,while SVM reached 98.33%and 97.91%,respectively.SVM was more robust than MLR in detecting yellow and rotten leaves of hydroponic.Thus,it was possible for abnormal hydroponic lettuce leaves detection by machine learning methods.

Detection of green apples in natural scenes based on saliency theory and Gaussian curve fitting

Green apple targets are difficult to identify for having similar color with backgrounds such as leaves.The primary goal of this study was to detect green apples in natural scenes by applying saliency detection and Gaussian curve fitting algorithm.Firstly,the image was represented as a close-loop graph with superpixels as nodes.These nodes were ranked based on the similarity to background and foreground queries to generate the final saliency map.Secondly,Gaussian curve fitting was carried out to fit the V-component in YUV color space in salient areas,and a threshold was selected to binarize the image.To verify the validity of the proposed algorithm,55 images were selected and compared with the common used image segmentation algorithms such as k-means clustering algorithm and FCM(Fuzzy C-means clustering algorithm).Four parameters including recognition ratio,FPR(false positive rate),FNR(false negative rate)and FDR(false detection rate)were used to evaluate the results,which were 91.84%,1.36%,8.16%and 4.22%,respectively.The results indicated that it was effective and feasible to apply this method to the detection of green apples in nature scenes.

Automatic detection of ruminant cows’ mouth area during rumination based on machine vision and video analysis technology

In order to realize the automatic monitoring of ruminant activities of cows,an automatic detection method for the mouth area of ruminant cows based on machine vision technology was studied.Optical flow was used to calculate the relative motion speed of each pixel in the video frame images.The candidate mouth region with large motion ranges was extracted,and a series of processing methods,such as grayscale processing,threshold segmentation,pixel point expansion and adjacent region merging,were carried out to extract the real area of cows’mouth.To verify the accuracy of the proposed method,six videos with a total length of 96 min were selected for this research.The results showed that the highest accuracy was 87.80%,the average accuracy was 76.46%and the average running time of the algorithm was 6.39 s.All the results showed that this method can be used to detect the mouth area automatically,which lays the foundation for automatic monitoring of cows’ruminant behavior.

Design and simulation of an integrated end-effector for picking kiwifruit by robot

The harvesting of fresh kiwifruit is a labor-intensive operation that accounts for more than 25%of annual production costs.Mechanized harvesting technologies are thus being developed to reduce labor requirements for harvesting kiwifruit.To improve the efficiency of a harvesting robot for picking kiwifruit,we designed an end-effector,which we report herein along with the results of tests to verify its operation.By using the established method of automated picking discussed in the literature and which is based on the characteristics of kiwifruit,we propose an automated method to pick kiwifruit that consists of separating the fruit from its stem on the tree.This method is experimentally verified by using it to pick clustered kiwifruit in a scaffolding canopy cultivation.In the experiment,the end-effector approaches a fruit from below and then envelops and grabs it with two bionic fingers.The fingers are then bent to separate the fruit from its stem.The grabbing,picking,and unloading processes are integrated,with automated picking and unloading performed using a connecting rod linkage following a trajectory model.The trajectory was analyzed and validated by using a simulation implemented in the software Automatic Dynamic Analysis of Mechanical Systems(ADAMS).In addition,a prototype of an end-effector was constructed,and its bionic fingers were equipped with fiber sensors to detect the best position for grabbing the kiwifruit and pressure sensors to ensure that the damage threshold was respected while picking.Tolerances for size and shape were incorporated by following a trajectory groove from grabbing and picking to unloading.The end-effector separates clustered kiwifruit and automatically grabs individual fruits.It takes on average 4–5 s to pick a single fruit,with a successful picking rate of 94.2%in an orchard test featuring 240 samples.This study shows the grabbing–picking–unloading robotic end-effector has significant potential to facilitate the harvesting of kiwifruit.

Assessment of canopy vigor information from kiwifruit plants based on a digital surface model from unmanned aerial vehicle imagery

Information about canopy vigor and growth are critical to assess the potential impacts of biotic or abiotic stresses on plant development.By implementing a Digital Surface Model(DSM)to imagery obtained using Unmanned Aerial Vehicles(UAV),it is possible to filter canopy information effectively based on height,which provides an efficient method to discriminate canopy from soil and lower vegetation such as weeds or cover crops.This paper describes a method based on the DSM to assess canopy growth(CG)as well as missing plants from a kiwifruit orchard on a plant-by-plant scale.The DSM was initially extracted from the overlapping RGB aerial imagery acquired over the kiwifruit orchard using the Structure from Motion(SfM)algorithm.An adaptive threshold algorithm was implemented using the height difference between soil/lower plants and kiwifruit canopies to identify plants and extract canopy information on a non-regular surface.Furthermore,a customized algorithm was developed to discriminate single kiwifruit plants automatically,which allowed the estimation of individual canopy cover fractions(fc).By applying differential fc thresholding,four categories of the CG were determined automatically:(i)missing plants;(ii)low vigor;(iii)moderate vigor;and(iv)vigorous.Results were validated by a detailed visual inspection on the ground,which rendered an overall accuracy of 89.5%for the method proposed to assess CG at the plant-by-plant level.Specifically,the accuracies for CG category(i)-(iv)were 94.1%,85.1%,86.7%,and 88.0%,respectively.The proposed method showed also to be appropriate to filter out weeds and other smaller non-plant materials which are extremely difficult to be distinguished by common colour thresholding or edge identification methods.

An adaptive segmentation method combining MSRCR and mean shift algorithm with K-means correction of green apples in natural environment

During the recognition and localization process of green apple targets,problems such as uneven illumination,occlusion of branches and leaves need to be solved.In this study,the multi-scale Retinex with color restoration(MSRCR)algorithm was applied to enhance the original green apple images captured in an orchard environment,aiming to minimize the impacts of varying light conditions.The enhanced images were then explicitly segmented using the mean shift algorithm,leading to a consistent gray value of the internal pixels in an independent fruit.After that,the fuzzy attention based on information maximization algorithm(FAIM)was developed to detect the incomplete growth position and realize threshold segmentation.Finally,the poorly segmented images were corrected using the K-means algorithm according to the shape,color and texture features.The users intuitively acquire the minimum enclosing rectangle localization results on a PC.A total of 500 green apple images were tested in this study.Compared with the manifold ranking algorithm,the K-means clustering algorithm and the traditional mean shift algorithm,the segmentation accuracy of the proposed method was 86.67%,which was 13.32%,19.82%and 9.23%higher than that of the other three algorithms,respectively.Additionally,the false positive and false negative errors were 0.58%and 11.64%,respectively,which were all lower than the other three compared algorithms.The proposed method accurately recognized the green apples under complex illumination conditions and growth environments.Additionally,it provided effective references for intelligent growth monitoring and yield estimation of fruits.

Fusion of machine vision technology and AlexNet-CNNs deep learning network for the detection of postharvest apple pesticide residues

Pesticide residue is an important factor that affects food safety.In order to achieve effective detection of pesticide residues in apples,a machine-vision-based segmentation algorithm and hyperspectral techniques were used to segment the foreground and background regions of the apple image.By calculating the roundness value and extracting the region with the highest roundness value in the connected region,a region of interest(ROI)maskwas created for the apple.Four pesticides(chlorpyrifos,carbendazimand two mixed pesticides)and an inactive control were used at the same concentration of 100 ppm(except for the control group),and the hyperspectral region of the corresponding sample image was extracted by obtaining the different types of pesticide residues in the ROI masks.To increase the diversity of the samples and to expand the dataset,Gaussianwhite noise with a varying signal-to-noise ratio was added to each of the hyperspectral images of the apple.The number of samples was increased from four types of 12 samples to four types of 72 samples,giving 4608 hyperspectral data images in each category.The structure and parameters of a convolutional neural network(CNN)were determined using theoretical analysis and experimental verification.All the extracted hyperspectral images of apples were normalized to 227×227×3 pixels as the input of the CNN network for pesticide residue detection.There were 18,432 sample data of four types for 72 samples.Of these,12,288 images were selected using a bootstrap sampling method as the training set,and 6144 as the test set,with no overlap.The test results showthatwhen the number of training epochswas 10,the accuracy of the test set detectionwas 99.09%,and the detection accuracy of the single-band average imagewas 95.35%.A comparison with traditional k-nearest neighbor(KNN)and support vectormachine classification algorithms showed that the detection accuracy for KNNwas 43.75%and the average time was 0.7645 s.These results demonstrate that our method is a small-sample,noncontact,fast,effective and low-cost technique that can provide effective pesticide residue detection in postharvest apples.

Principles, developments and applications of laser-induced breakdown spectroscopy in agriculture: A review

Considering the diversity of soil contents,quality and usability,a systematic scientific study on the elemental and chemical composition(major and minor nutrients elements,trace elements,heavy metals,etc.)of soil is very important.Rapid and accurate detection and prevention of soil contamination(mainly in pollutants of heavy metals)is deemed to be a concerned and serious central issue inmodern agriculture and agricultural sustainable development.In order to study the chemical composition of soil,laser induced breakdown spectroscopy(LIBS)has been applied recently.LIBS technology,a kind of atomic emission spectroscopy,is regarded as a future“Superstar”in the field of chemical analysis and green analytical techniques.In this work,the research achievements and trends of soil elements detection based on LIBS technology were reviewed.The structural composition and operating principle of LIBS systemwas briefly introduced.The paper offered a reviewof LIBS applications,including detection and analysis of major element,minor nutrient element and heavy metal element.Simultaneously,LIBS applications to analysis of the soil related materials,plants-related issues(nutrients,pesticide residues,and plants disease)were briefly summarized.The research tendency and developing prospects of LIBS in agriculture were presented at last.

Image dehazing based on dark channel prior and brightness enhancement for agricultural monitoring

Obtaining clear and true images is a basic requirement for agricultural monitoring.However,under the influence of fog,haze and other adverse weather conditions,captured images are usually blurred and distorted,resulting in the difficulty of target extraction.Traditional image dehazing methods based on image enhancement technology can cause the loss of image information and image distortion.In order to address the above-mentioned problems caused by traditional image dehazing methods,an improved image dehazing method based on dark channel prior(DCP)was proposed.By enhancing the brightness of the hazed image and processing the sky area,the dim and un-natural problems caused by traditional image dehazing algorithms were resolved.Ten different test groups were selected from different weather conditions to verify the effectiveness of the proposed algorithm,and the algorithm was compared with the commonly-used histogram equalization algorithm and the DCP method.Three image evaluation indicators including mean square error(MSE),peak signal to noise ratio(PSNR),and entropy were used to evaluate the dehazing performance.Results showed that the PSNR and entropy with the proposed method increased by 21.81%and 5.71%,and MSE decreased by 40.07%compared with the original DCP method.It performed much better than the histogram equalization dehazing method with an increase of PSNR by 38.95%and entropy by 2.04%and a decrease of MSE by 84.78%.The results from this study can provide a reference for agricultural field monitoring.

Physical and mechanical properties of hydroponic lettuce for automatic harvesting

To design an automatic harvesting machine for hydroponic lettuce(Lactuca sativa L.),physical and mechanical properties of hydroponic lettuce were investigated and analyzed.Moisture content of stem,root and leaf,geometric characteristics,pulling force,and root cutting force were studied for harvesting hydroponic lettuce.The pulling force was examined by a tensile experiment,while the root cutting force was investigated by a shear experiment on the electronic universal testing machine.The moisture content of hydroponic lettuce was obtained by direct drying.Experiment data were processed using regression analysis and mathematical statistics method.A regression equation and the law of numerical distribution were obtained.The results showed that the geometric size of different hydroponic lettuce had little difference,and the distribution of physical parameters was concentrated.Moisture content was found statistically similar in stem and root(around 91%),while the highest moisture content was found in the leaf of 95.73%.The root cutting force decrease with the increase of cutting speed and decrease with the cutting position move downward.The minimum average root cutting force in the experiment was 1.41 N.The average pulling force was 13 N.This study provides adequate theoretical support for the design of the automatic harvesting machine of hydroponic lettuce.

Kinetic models of peroxidase activity in potato leaves infected with late blight based on hyperspectral data

Potato late blight,which is caused by Phytophthorainfestans(Mont.)de Bary,is a worldwide devastating disease for potato.It decreased yields of potato and caused unpredictable losses all over the world.Various simple statistical methods and forecasting models have been developed to predict and manage potato late blight.Meanwhile,there is a rising need to develop prediction models reflecting peroxidase(POD)activity,which is an important health index that varies with infection and correlated with stress resistance in plants.Thus,the aim of this research was to develop kinetic models to predict POD activity.Infection-induced changes in potato leaves stored in an artificial climate chest at 25°C were analyzed using hyperspectroscopy.Four prediction models were developed by using linear partial least squares(PLS)and nonlinear support vector machine(SVM)methods based on the full spectrum and effective wavelengths.The effective wavelengths were selected by the successive projection algorithm(SPA).In this study,the prediction model developed by means of SPA-SVM method obtained the best performance,with a Rp(correlation coefficient of prediction)value of 0.923 and a RMSEp(root mean square error of prediction)value of 24.326.Five-order kinetics models according to the prediction model were developed,and late blight disease can be predicted using this model.This study provided a theoretical basis for the prediction of latencies of late blight.

Automatic monitoring method of cow ruminant behavior based on spatio-temporal context learning

Automatic monitoring of cow rumination has great significance in the development of modern animal husbandry.In order to solve the problem of high real-time requirement of ruminant behavior monitoring,a tracking method based on STC(Spatio-Temporal Context)learning was carried out.On the basis of cow’s mouth region extraction,the spatial context model between target object and its local surrounding background was built based on their spatial correlations by solving the deconvolution problem,and the learned spatial context model was used to update the STC learning model for the next frame.Tracking in the next frame was formulated by computing a confidence map as a convolution problem that integrates the STC learning information,and the best object location could be estimated by maximizing the confidence map.Then the target scale was estimated based on the confidence evaluation.Finally,accurate tracking of the mouth movement trajectory was realized.To verify the effectiveness of the proposed method,the performance of the algorithm was tested using 20 video sequences.Besides,the tracking results were compared with the Mean-shift algorithm.The results showed that the average success rate of STC learning monitoring algorithm was 85.45%,which was 9.45%higher than the Mean-shift algorithm,the detection rate of STC learning monitoring algorithm was 18.56 s per video,which was 22.08%higher than that of the Mean-shift algorithm.The results showed that the fast tracking method based on STC learning monitoring algorithm is effective and feasible.

Automatic tracking of multi-fruit targets with oscillation under natural growth conditions

It is important for intelligent orchards to be able to achieve automatic monitoring of fruit growth information within a natural growing environment.The issue of how to track green and oscillating fruits under the influence of wind and farming operations is a key aspect of monitoring of the growth state of the fruit.In order to realize the accurate tracking of green fruit targets,a new method based on target tracking is proposed.First,an optical flow method is applied to realize the automatic detection of green fruit targets,and this lays the foundation for the accurate and automatic tracking of these targets.Then,Kalman and kernelized correlation filter(KCF)algorithms are applied to achieve multi-target tracking and prediction.In order to verify the performance of these different algorithms on various types of green fruit targets,experiments were carried out based on nine video sequences.The experimental results for the tracking of single,double and triple green fruit targets show that the average tracking success rates of the Kalman algorithm are 88.15%,82.30%and 53.10%,respectively,and those of the KCF algorithm are 94.07%,87.35%and 61.46%,respectively,meaning that the average tracking results from KCF are 5.92%,5.05%and 8.36%higher than those from the Kalman algorithm.The time consumed is also reduced by 35.40%,36.27%and 40.86%,respectively.The results show that it is feasible to apply the KCF algorithm to the tracking of green fruit targets.

Corn ear test using SIFT-based panoramic photography and machine vision technology

Corn ear test is important to modern corn breeding.The test indexesmainly include lengths,radiuses,rows and numbers of corn ears and the kernels they bear,which can benefit the study on breeding new and fine corn varieties.These corn traits are often collected by traditional manual measurement,which is difficult to meet the needs of high throughput corn ear test.In this study,image sequences of corn ear samples were captured by building a panoramic photography collecting system.And then,to get the lengths and radiuses indexes,the corn area images were processed based on Lab color space and adaptive threshold segmentation.The sequence images were then matched and the panoramic image of a corn surface were extracted using Scale-invariant feature transform(SIFT).Finally,by using Exponential transformation(ETR)and Sobel-Hough algorithm,ears and rows indexes were acquired.Test results showed that the accuracy of the radiuses and lengths were 93.84%and 94.53%,respectively.Meanwhile,the accuracy of kernels and rows indexes were 98.12%and 96.14%,whichwere 4.03%and 7.25%higher than that of common mosaiced panoramic image.And the accuracy of kernel area and length-width ratio were 95.36%and 97.42%,respectively.All the results showed that the proposed method can be used for corn ear test effectively.

Effects of cutter parameters on shearing stress for lettuce harvesting using a specially developed fixture

To investigate the optimal parameters combination of reciprocating cutter for harvesting hydroponic lettuce automatically,a shear fixture was designed for cutting lettuce stems on a universal materials tester.Effects of blade distance,sliding cutting angle,skew cutting angle,and shearing angle on shearing stress were investigated in this study.The orders of the significance of a single factor and double factors were analyzed using the response surface methodology(RSM).A scanning electron microscope was used to observe the microstructure of the lettuce stem to analyze the shearing characteristics at the microscopic level.The RSM results showed that the order of significance for single factors was(i)sliding cutting angle,(ii)shearing angle,(iii)skew cutting angle,and(iv)blade distance.The sliding cutting angle had a highly significant influence on the shearing stress.The order of significance for double factors was(i)blade distance and shearing angle,(ii)sliding cutting angle and skew cutting angle,and(iii)the sliding cutting angle and shearing angle.A quadratic model of the factors and shearing stress was built according to the response-surface results.The optimized combination of factors that gives the minimum shearing stress was observed that it reduced 69.9%of the maximum shearing stress value.This research can provide a reference for designing lettuce-cutting devices.

Segmentation of field grape bunches via an improved pyramid scene parsing network

With the continuous expansion of wine grape planting areas,the mechanization and intelligence of grape harvesting have gradually become the future development trend.In order to guide the picking robot to pick grapes more efficiently in the vineyard,this study proposed a grape bunches segmentation method based on Pyramid Scene Parsing Network(PSPNet)deep semantic segmentation network for different varieties of grapes in the natural field environments.To this end,the Convolutional Block Attention Module(CBAM)attention mechanism and the atrous convolution were first embedded in the backbone feature extraction network of the PSPNet model to improve the feature extraction capability.Meanwhile,the proposed model also improved the PSPNet semantic segmentation model by fusing multiple feature layers(with more contextual information)extracted by the backbone network.The improved PSPNet was compared against the original PSPNet on a newly collected grape image dataset,and it was shown that the improved PSPNet model had an Intersection-over-Union(IoU)and Pixel Accuracy(PA)of 87.42%and 95.73%,respectively,implying an improvement of 4.36%and 9.95%over the original PSPNet model.The improved PSPNet was also compared against the state-of-the-art DeepLab-V3+and U-Net in terms of IoU,PA,computation efficiency and robustness,and showed promising performance.It is concluded that the improved PSPNet can quickly and accurately segment grape bunches of different varieties in the natural field environments,which provides a certain technical basis for intelligent harvesting by grape picking robots.

An improved binocular localization method for apple based on fruit detection using deep learning

Apple picking robot is now being developed as an alternative to hand picking due to a great demand for labor during apple harvest season.Accurate detection and localization of target fruit is necessary for robotic apple picking.Detection accuracy has a great influence on localization results.Although current researches on detection and localization of apples using traditional image algorithms can obtain good results under laboratory conditions,it is difficult to accurately detect and locate objects in natural field with complex environments.With the rapid development of artificial intelligence,accuracy of apple detection based on deep learning has been significantly improved.Therefore,a deep learningbased method was developed to accurately detect and locate the position of fruit.For different localization methods,binocular localization is a widely used localization method for its bionic principle and lower equipment cost.Hence,this paper proposed an improved binocular localization method for apple based on fruit detection using deep learning.First,apples of binocular images were detected by Faster R-CNN.After that,a segmentation based on chromatic aberration and chromatic aberration ratio was applied to segment apple and background pixels in bounding box of detected fruit.Furthermore,template matching with parallel polar line constraint was used to match apples in left and right images.Finally,two feature points on apples were selected to directly calculate three dimensional coordinates of feature points with the binocular localization principle.In this study,Faster R-CNN achieved an AP of 88.12%with an average detection speed of 0.32 s for an image.Meanwhile,standard deviation and localization precision of depth of two feature points on each apple were calculated to evaluate localization.Results showed that the average standard deviation and the average localization precision of 76 groups of datasets were 0.51 cm and 99.64%,respectively.Results indicated that the proposed improved binocular localization method is promising for fruit localization。

Mechanized technologies for scaffolding cultivation in the kiwifruit industry:A review

The success of organic and green agricultural fruit production depends on quality and cost.As the kiwifruit industry becomes ever more commercialized,it is in the interests of the industry to mechanize production,which can promote industrialization and improve industrial value and market prospects.Currently,New Zealand,Italy,Chile,and China carry out research into the mechanism of kiwifruit production.This review describes in detail the current state of the art of pollination,harvesting and grading equipment,including detection and identification,non-destructive end effector,harvesting robots and grading devices.Process technologies that include artificial pollination,harvest mechanization,grading and standardization of production problems are analysed and compared.These problems directly affect the quality of kiwifruit products.Finally,to solve the various problems that the kiwifruit industry experiences,it is necessary to accelerate the development of mechanized kiwifruit production,realize the mechanization of information acquisition and standardization in order to advance precision agriculture and agricultural wisdom for the future.Mechanization of the kiwifruit industry must adapt to adjustments in how China’s economic structure develops.

Model for tomato photosynthetic rate based on neural network with genetic algorithm

A photosynthetic rate model provides a theoretical basis for fine-grained control of light,and has become the key component to determine the effectiveness of light-controlled environments.Therefore,it is critical to identify an intelligent algorithm that can be used to build an efficient and precise photosynthetic rate model.Depending on the initial weights of a BP(Back Propagation)neural network algorithm for arbitrary random numbers,the establishment of a regressive prediction model can be easily trapped in a partially-flat area.Existing photosynthetic rate models based on neural networks are facing problems such as a slow convergence speed and a long training time,and this study presents a photosynthetic rate model of a heuristic neural network for tomatoes based on a genetic algorithm to address the above problems.The performance of the model can be effectively improved using a genetic algorithm to optimize the initial weights.A multi-factor nesting experiment was firstly conducted to obtain 825 groups of tomato seedling photosynthesis rate test data in the foundation,and the photosynthetic rate model of the heuristic neural network for the tomato is established through BP network structure construction and data preprocessing.The genetic algorithm was used to optimize the network weights and threshold,and the LM(Levenberg-Marquardt)training method for network training.On this basis,the training performance and precision of the photosynthetic rate prediction models can be further compared with the genetic neural network model and the neural network model.The test results have shown that the training effects and accuracy of the genetic neural network prediction model of the photosynthetic rate were better than those of the neural network prediction model.The correlation coefficient between the model predicted data and the measured data is 0.987,and the absolute error of the photosynthetic rate is less than±0.5μmol/(m^(2)·s).