An Adaptive Vision Navigation Algorithm in Agricultural IoT System for Smart Agricultural Robots

As the agricultural internet of things(IoT)technology has evolved,smart agricultural robots needs to have both flexibility and adaptability when moving in complex field environments.In this paper,we propose the concept of a vision-based navigation system for the agricultural IoT and a binocular vision navigation algorithm for smart agricultural robots,which can fuse the edge contour and the height information of rows of crop in images to extract the navigation parameters.First,the speeded-up robust feature(SURF)extracting and matching algorithm is used to obtain featuring point pairs from the green crop row images observed by the binocular parallel vision system.Then the confidence density image is constructed by integrating the enhanced elevation image and the corresponding binarized crop row image,where the edge contour and the height information of crop row are fused to extract the navigation parameters(θ,d)based on the model of a smart agricultural robot.Finally,the five navigation network instruction sets are designed based on the navigation angleθand the lateral distance d,which represent the basic movements for a certain type of smart agricultural robot working in a field.Simulated experimental results in the laboratory show that the algorithm proposed in this study is effective with small turning errors and low standard deviations,and can provide a valuable reference for the further practical application of binocular vision navigation systems in smart agricultural robots in the agricultural IoT system.

Path planning for agricultural robots in wild livestock farm environments

Path planning for field agricultural robots must satisfy several criteria:establishing feeding routes,maintaining gentle slopes,approaching multiple livestock observation points,ensuring timely environmental monitoring,and achieving high efficiency.The complex terrain of outdoor farming areas poses a challenge.Traditional A*algorithms,which generate only the shortest path,fail to meet these requirements and often produce paths that lack smoothness.Therefore,identifying the most suitable path,rather than merely the shortest one,is essential.This study introduced a path-planning algorithm tailored to field-based livestock farming environments,building upon the traditional A*algorithm.It constructed a digital elevation model,integrated an artificial potential field for evaluating multiple target points,calculated terrain slope,optimized the search neighborhood based on robot traversability,and employed Bézier curve segmentation for path optimization.This method segmented the path into multiple curves by evaluating the slopes of the lines connecting adjacent nodes,ensuring a smoother and more efficient route.The experimental results demonstrate its superiority to traditional A^(*),ensuring paths near multiple target points,significantly reducing the search space,and resulting in over 69.4%faster search speeds.Bézier curve segmentation delivers smoother paths conforming to robot trajectories.

Autonomous trajectory tracking control method for an agricultural robotic vehicle

To address the nonlinearities and external disturbances in unstructured and complex agricultural environments,this paper investigates an autonomous trajectory tracking control method for agricultural ground vehicles.Firstly,this paper presents the design and implementation of a lightweight,modular two-wheeled differential drive vehicle equipped with two drive wheels and two caster wheels.The vehicle comprises drive wheel modules,passive wheel modules,battery modules,a vehicle frame,a sensor system,and a control system.Secondly,a novel robust trajectory tracking method was proposed,utilizing an improved pure pursuit algorithm.Additionally,an Online Particle Swarm Optimization Continuously Tuned PID(OPSO-CTPID)controller was introduced to dynamically search for optimal control gains for the PID controller.Simulation results demonstrate the superiority of the improved pure pursuit algorithm and the OPSO-CTPID control strategy.To validate the performance,the vehicle was integrated with a seeding and fertilizing machine to realize autonomous wheat seeding in an agricultural environment.Experimental outcomes reveal that the vehicle of this study completed a seeding operation exceeding 1 km in distance.The proposed method can robustly and smoothly track the desired trajectory with an accuracy of less than 10 cm for the root mean square error(RMSE)of the curve and straight lines,given a suitable set of parameters,meeting the requirements of agricultural applications.The findings of this study hold significant reference value for subsequent research on trajectory tracking algorithms for ground-based agricultural robots.

Rapid development methodology of agricultural robot navigation system working in GNSS-denied environment

Robotic autonomous operating systems in global n40avigation satellite system(GNSS)-denied agricultural environments(green houses,feeding farms,and under canopy)have recently become a research hotspot.3D light detection and ranging(LiDAR)locates the robot depending on environment and has become a popular perception sensor to navigate agricultural robots.A rapid development methodology of a 3D LiDAR-based navigation system for agricultural robots is proposed in this study,which includes:(i)individual plant clustering and its location estimation method(improved Euclidean clustering algorithm);(ii)robot path planning and tracking control method(Lyapunov direct method);(iii)construction of a robot-LiDAR-plant unified virtual simulation environment(combination use of Gazebo and SolidWorks);and(vi)evaluating the accuracy of the navigation system(triple evaluation:virtual simulation test,physical simulation test,and field test).Applying the proposed methodology,a navigation system for a grape field operation robot has been developed.The virtual simulation test,physical simulation test with GNSS as ground truth,and field test with path tracer showed that the robot could travel along the planned path quickly and smoothly.The maximum and mean absolute errors of path tracking are 2.72 cm,1.02 cm;3.12 cm,1.31 cm,respectively,which meet the accuracy requirements of field operations,establishing the effectiveness of the proposed methodology.The proposed methodology has good scalability and can be implemented in a wide variety of field robot,which is promising to shorten the development cycle of agricultural robot navigation system working in GNSS-denied environment.

Research and development in agricultural robotics:A perspective of digital farming

Digital farming is the practice of modern technologies such as sensors,robotics,and data analysis for shifting from tedious operations to continuously automated processes.This paper reviews some of the latest achievements in agricultural robotics,specifically those that are used for autonomous weed control,field scouting,and harvesting.Object identification,task planning algorithms,digitalization and optimization of sensors are highlighted as some of the facing challenges in the context of digital farming.The concepts of multi-robots,human-robot collaboration,and environment reconstruction from aerial images and ground-based sensors for the creation of virtual farms were highlighted as some of the gateways of digital farming.It was shown that one of the trends and research focuses in agricultural field robotics is towards building a swarm of small scale robots and drones that collaborate together to optimize farming inputs and reveal denied or concealed information.For the case of robotic harvesting,an autonomous framework with several simple axis manipulators can be faster and more efficient than the currently adapted professional expensive manipulators.While robots are becoming the inseparable parts of the modern farms,our conclusion is that it is not realistic to expect an entirely automated farming system in the future.

Simulation software and virtual environments for acceleration of agricultural robotics:Features highlights and performance comparison

Research efforts for development of agricultural robots that can effectively perform tedious field tasks have grown significantly in the past decade.Agricultural robots are complex systems that require interdisciplinary collaborations between different research groups for effective task delivery in unstructured crops and plants environments.With the exception of milking robots,the extensive research works that have been carried out in the past two decades for adaptation of robotics in agriculture have not yielded a commercial product to date.To accelerate this pace,simulation approach and evaluation methods in virtual environments can provide an affordable and reliable framework for experimenting with different sensing and acting mechanisms in order to verify the performance functionality of the robot in dynamic scenarios.This paper reviews several professional simulators and custom-built virtual environments that have been used for agricultural robotic applications.The key features and performance efficiency of three selected simulators were also compared.A simulation case study was demonstrated to highlight some of the powerful functionalities of the Virtual Robot Experimentation Platform.Details of the objects and scenes were presented as the proof-of-concept for using a completely simulated robotic platform and sensing systems in a virtual citrus orchard.It was shown that the simulated workspace can provide a configurable and modular prototype robotic system that is capable of adapting to several field conditions and tasks through easy testing and debugging of control algorithms with zero damage risk to the real robot and to the actual equipment.This review suggests that an open-source software platform for agricultural robotics will significantly accelerate effective collaborations between different research groups for sharing existing workspaces,algorithms,and reusing the materials.

Development status and trend of agricultural robot technology

In the face of the contradiction between the increasing demand for agricultural products and the sharp reduction of agricultural resources and labor force,agricultural robot technology is developing explosively on the basis of decades of technical and industrial exploration.In view of the complexity and particularity of the development of agricultural robot technology,it is of great value to summarize its development characteristics and make reasonable judgments on its development trend.In this paper,the type of agricultural robot systems was first discussed.From the classification of agricultural robot systems,the development of main types of monitoring robots,non-selective and selective working robots for crop farming,livestock and poultry farming and aquaculture were introduced in detail.Then the scientific research,core technology,and commercialization of different types of agricultural robots were summarized.It is believed that navigation in complex agricultural environments,damage-free robot-crop interaction,and agronomy-robot fusion have high scientific value and significance to promote the revolutionary advances in agricultural robot technology.The characteristics of inter-discipline between agricultural robot technology and new materials,artificial intelligence,bionics,agronomy are research focus.The fast damage-free operation,autonomous navigation for complex environments,target detection for complex backgrounds,and special design for agricultural robots are considered to be the key technology of agricultural robot development,and the development path is given.Finally,robot-crop interaction simulation,big data support,and artificial intelligence are regarded as paths to realize the breakthrough of key agricultural robot technologies.The summary and prospect of this paper are of positive significance to promote the overall development of agricultural robot technology.

Slipping detection and control in gripping fruits and vegetables for agricultural robot

The minimum gripping force applied is expected to prevent objects from mechanical damage when an agricultural robot is applied to handle and manipulate fruits and vegetables.In this research,a sensitive slipping sensor was developed with a piezo resistor to control the griping force of the agricultural robot.Firstly,an output of the slipping sensor was analyzed in a frequency domain by using a short time Fourier transform.Then rules for discriminating slipping signal from the output of a slipping sensor were proposed based on detail coefficients of discrete wavelet transform.Finally,a controller based on adaptive Neuro-Fuzzy inference system was developed to adjust the grasping force of the agricultural robot in real time.The detail coefficients and the normal gripping force were applied as input of the controller,and Fuzzy rules were simplified through subtractive clustering.With a two-finger end-effector of the agricultural robot,the experimental results showed that the slipping signal could be effectively extracted regardless of change in the normal gripping force,and the gripping force had been controlled successfully when grasping tomatoes and apples.This method was a promising way to optimize the gripping force of the agricultural robot grasping the fruits and vegetables.

Review of Simultaneous Localization and Mapping Technology in the Agricultural Environment

Simultaneous localization and mapping(SLAM)is one of the most attractive research hotspots in the field of robotics,and it is also a prerequisite for the autonomous navigation of robots.It can significantly improve the autonomous navigation ability of mobile robots and their adaptability to different application environments and contribute to the realization of real-time obstacle avoidance and dynamic path planning.Moreover,the application of SLAM technology has expanded from industrial production,intelligent transportation,special operations and other fields to agricultural environments,such as autonomous navigation,independent weeding,three-dimen-sional(3D)mapping,and independent harvesting.This paper mainly introduces the principle,sys-tem framework,latest development and application of SLAM technology,especially in agricultural environments.Firstly,the system framework and theory of the SLAM algorithm are introduced,and the SLAM algorithm is described in detail according to different sensor types.Then,the devel-opment and application of SLAM in the agricultural environment are summarized from two aspects:environment map construction,and localization and navigation of agricultural robots.Finally,the challenges and future research directions of SLAM in the agricultural environment are discussed.

Image moments-based visual servoing control of bagged agricultural materials handling robot

Manual handling is less efficient and sometimes even hazardous to humans in many areas,for example,agriculture.Using robots in those areas not only avoids human contact with such dangerous agricultural materials but also improves working efficiency.The motion of a robot is controlled using a technique called visual servoing that uses feedback information extracted from a vision sensor.In this study,a visual servoing method was proposed based on learning features and image moments for 3D targets to solve the problem of image moment-based visual servoing.A Gaussian process regression model was used to map the relationship between the image moment invariants and the rotational angles around the X-and Y-axes of the camera frame(denoted asγandβ).To obtain maximal decoupled structure and minimal nonlinearities of the image Jacobian matrix,it was assumed two image moment features,which are linearly proportional toγandβ.In addition to the other four standardized image moment features,a 6-DOF image moment-based visual servoing controller for the agricultural material handling robot was designed.Using this method,the problem of visual servoing task failure due to the singularity of the Jacobian matrix was solved,and it also had a better convergence effect for the part of the target image beyond the field of view and large displacement visual servoing system.The proposed algorithm was validated by carrying out experiments tracking bagged flour in a six-degree-of-freedom robotic system.The final displacement positioning accuracy reached the millimeter level and the direction angle positioning accuracy reached the level of 0.1°.The method still has a certain convergence effect when the target image is beyond the field of view.The experimental results have been presented to show the adequate behavior of the presented approach in robot handling operations.It provides reference for the application of visual servoing technology in the field of agricultural robots and has important theoretical significance and practical value.

Review of smart robots for fruit and vegetable picking in agriculture

The fruit and vegetable picking has posed a great challenge on the production and markets during the harvest season.Manual picking cannot fully meet the rapid requirements of each market,mainly due to the high labor-intensive and time-consuming tasks,even the aging and shortage of agricultural labor force in recent years.Alternatively,smart robotics can be an efficient solution to increase the planting areas for the markets in combination with changes in cultivation,preservation,and processing technology.However,some improvements still need to be performed on these picking robots.To document the progress in and current status of this field,this study performed a bibliometric analysis.This analysis evaluated the current performance characteristics of various fruit and vegetable picking robots for better prospects in the future.Five perspectives were proposed covering the robotic arms,end effectors,vision systems,picking environments,and picking performance for the large-scale mechanized production of fruits and vegetables in modern agriculture.The current problems of fruit and vegetable picking robots were summarized.Finally,the outlook of the fruit and vegetable picking robots prospected from four aspects:structured environment for fruit planting,the algorithm of recognition and positioning,picking efficiency,and cost-saving picking robots.This study comprehensively assesses the current research status,thus helping researchers steer their projects or locate potential collaborators.

Design and performance of a robotic arm for farm use

Smart technology which is the backbone of high-efficiency production opens a new horizon in sustainable agriculture.Nowadays,harvesting the heavy-weight crops is considered an arduous job,specifically in Japan which has faced a serious labor shortage in agricultural fields.In this study,a development procedure and evaluation of a 4-degrees-of-freedom articulated robotic arm is presented,and it provides an appropriate infrastructure to develop a smart harvesting robotic system for heavy-weight crops such as pumpkin,watermelon,melon,and cabbage.This robotic arm designed as an actuating unit of a robot tractor for the agricultural outdoor environment.In this regard,different degree of freedom was evaluated under consideration of economic and technical indexes to find an optimized mechanism.The controlling algorithm of the system was developed by consideration of kinematic and dynamic aspects of the real-world condition.A special harvesting methodology was developed based on optimum harvesting conditions.A controlling unit was developed by using PLC system.Experimental performance,accuracy,payload per weight,and repeatability of the system were measured.The payload per weight,overall average accuracy,and overall average repeatability of the robot were 0.21,1.85 mm,and±0.51 mm,respectively.The results indicated that the developed system had a front access,harvesting length,and workspace volume of 2.024 m,1.36 m,and 8.27 m^(3),respectively.One of the significant advantages of the proposed robotic arm is its capability to use in different industries with minimum modifications.

Fuzzy Tuned PID Controller for Envisioned Agricultural Manipulator

The implementation of image-based phenotyping systems has become an important aspect of crop and plant science research which has shown tremendous growth over the years. Accurate determination of features using images requires stable imaging and very precise processing. By installing a camera on a mechanical arm driven by motor, the maintenance of accuracy and stability becomes non-trivial. As per the state-of-the-art, the issue of external camera shake incurred due to vibration is a great concern in capturing accurate images, which may be induced by the driving motor of the manipulator. So, there is a requirement for a stable active controller for sufficient vibration attenuation of the manipulator. However, there are very few reports in agricultural practices which use control algorithms. Although, many control strategies have been utilized to control the vibration in manipulators associated to various applications, no control strategy with validated stability has been provided to control the vibration in such envisioned agricultural manipulator with simple low-cost hardware devices with the compensation of non-linearities. So, in this work, the combination of proportional-integral-differential(PID) control with type-2 fuzzy logic(T2-F-PID) is implemented for vibration control. The validation of the controller stability using Lyapunov analysis is established. A torsional actuator(TA) is applied for mitigating torsional vibration, which is a new contribution in the area of agricultural manipulators. Also, to prove the effectiveness of the controller, the vibration attenuation results with T2-F-PID is compared with conventional PD/PID controllers, and a type-1 fuzzy PID(T1-F-PID) controller.

Gait analysis of goat at different slopes and study on biomimetic walking mechanism

To study the walking status of a goat on slope and the walking mechanics,a comparative analysis of the walking gait mode of a goat on different slopes was conducted.The uphill and downhill walking processes on different slopes(6°,18°and 36°)were recorded by a high speed video system.The experimental image results were processed and analyzed via SigmaScan software and MATLAB software.The characteristic parameters of locomotion gait on the different slopes and the angle change curves of each goat leg in the process of slope movement indicated that the goat performed tetrapod gait,static gait,ipsilateral gait and trot gait on different slopes.With the increase of gradient in the uphill process,the corresponding load factors of each leg were 0.65±0.15,0.75 and 0.645±0.205,whereas those in the downhill process were 0.70±0.08,0.66±0.06,and 0.685±0.125.Results showed that the load factors of each leg are higher than 0.5.The foreleg angle ofa(the angle of wrist joints),which ranges from 100°to 130°,is suitable for different slopes.However,the angle ofb(the angle between the thigh and the forward direction in the walking process of goat),which ranges from 99°to 109°,is suitable for the 6°slope,whereas the angle ranging from 46°to 91°is suitable for the 18°slope and 36°slope.For the hind leg,the angle ofa,which ranges from 105°to 153°,is suitable for the different slopes.The angle ofb,which ranges from 128°to 150°,is suitable for the different slopes.The research can provide a theoretical basis and experimental data for the design of biomimetic agricultural slope walking mechanism.

Analysis of the track system in bumpy unstructured hard road environment by vibration test

The complex terrain environment in the hilly land directly affects the operational reliability of agricultural robots.In order to study the impact of road irregularity on walking chassis vibration,the 3CYLZ-750 remote-controlled weeding machine which is applied to orchards was taken as the object of study,and the rear roller was selected as the object of observation to reveal the rules under which the vibration of the track chassis changes as there is a sudden change in road surface elevation.A column-type test-to-pass method based on unit excitation was proposed in this study.The excitation behavior and action process were analyzed by category.A critical acceleration prediction model was built and verified by virtual simulation and hard road surface excitation testing.The results showed that at the forward velocity of 0-2.5 km/h and exciter height of 20-100 mm,the vertical vibration acceleration of the target roller was significantly affected by Track Contact Point Centrifugal Acceleration(TCPCA).As TCPCA increased,the change rate of vertical vibration acceleration decreased,reaching a minimum of[−13.8,28.8];as TCPCA decreased,the vertical vibration acceleration tended to increase positively at a maximum variation range of[−13.3,42.2].The measured and simulated macroscopic change rules were consistent with the theoretical analysis,further verifying the correctness of variable extraction,and providing a research basis for the accurate modification and improvement of the model.The research conclusions can lay a theoretical foundation for analyzing the walking reliability of the track chassis,and provide a design basis and technical support for the development of a tracked agricultural robot chassis for the hilly land in the future.