Sliding Mode Variable Structure Control for Visual Servoing System

A visual servoing tracking controller is proposed based on the sliding mode control theory in order to achieve strong robustness against parameter variations and external disturbances. A sliding plane with time delay compensation is presented by the pre-estimate of states. To reduce the chattering of the sliding mode controller, a modified exponential reaching law and hyperbolic tangent function are applied to the design of visual controller and robot joint controller. Simulation results show that the visual servoing control scheme is robust and has good tracking performance.

Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator

This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s sixdimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.

SVM-based Identification and Un-calibrated Visual Servoing for Micro-manipulation

This paper presents an improved support vector machine （SVM） algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set＇s discernible matrix is proposed to identify and classify micro-targets. To avoid the complicated calibration for intrinsic parameters of camera, an improved Broyden＇s method is proposed to estimate the image Jacobian matrix which employs Chebyshev polynomial to construct a cost function to approximate the optimization value. Finally, a visual controller is designed for a robotic micromanipulation system. The experiment results of micro-parts assembly show that the proposed methods and algorithms are effective and feasible.

Robust visual servoing based Chinese calligraphy on a humanoid robot

A robust visual servoing system is investigated on a humanoid robot which grasps a brush in Chinese calligraphy task.The system is implemented based on uncalibrated visual servoing controller utilizing Kalman-Bucy filter,with the help of an object detector by continuously adaptive MeanShift(CAMShift) algorithm.Under this control scheme,a humanoid robot can satisfactorily grasp a brush without system modeling.The proposed method is shown to be robust and effective through a Chinese calligraphy task on a NAO robot.

CONTROL SCHEMES FOR CMAC NEURAL NETWORK-BASED VISUAL SERVOING

In IBVS (image based visual servoing), the error signal in image space should be transformed into the control signal in the input space quickly. To avoid the iterative adjustment and complicated inverse solution of image Jacobian, CMAC (cerebellar model articulation controller) neural network is inserted into visual servo control loop to implement the nonlinear mapping. Two control schemes are used. Simulation results on two schemes are provided, which show a better tracking precision and stability can be achieved using scheme 2.

Path Planning of Planar Kinked Line Seam by Visual Servoing for Robotic Welding

Welding path planning can substitute for the manual teaching process of the robot and can promote the autonomous level of the robotic welding. A path planning method by visual servoing was presented, in which the optimal angle of charge-coupled device (CCD) camera was also planned. Aiming at planning two forms of kinked line seams, obtuse angle seam and right angle seam, a practicable solution was put forward. In this solution, the intersection of two adjacent straight segments is detected in each local seam image, and if intersection is found, the seam errors are calculated using the next straight segment. The experimental results show that kinked line seam can be well planned using this solution.

Study on algorithm of dynamic uncalibrated eye-in-hand visual servoing system

Currently, most visual servoing system must be calibrated, while it is impossible to calibrate cameras and robot models precisely in industrial practice, so a novel dynamic uncalibrated eye-in-hand visual servoing system of tracking a moving target is proposed. The method does not require calibration of camera and robot kinematic models. Vision guided algorithm for tracking dynamic image is developed through minimizing nonlinear objective function. For the large residual has not been approximated in dynamic environment and the change of composite image Jacobian with time increment has not been computed in visual servoing system now,large residuals are dynamic approximated and the change of composite image Jacobian at each iterative step is computed. Simulation results demonstrate the validity of these approaches.

Uncalibrated visual servoing design for competitive networked robots

In this paper,a method with an eye-in-hand configuration is developed to hit targets during visual tracking for the TLS(Tele-Light Saber) game.It is not necessary to calibrate camera parameters and predict the trajectory of the moving object.Firstly,the expression of the image Jacobian matrix for the eye-in-hand configuration is proposed,and then an update law is designed to estimate the image Jacobian online.Furthermore,a control scheme is presented and the Lyapunov method is employed to prove asymptotic convergence of image errors.No assumption for the moving objects is needed.Finally,both simulation and experimental results are shown to support the approach in this paper.

A Stable Switch Method Based on Fusion in Uncalibrated Visual Servoing

Stable switch control between multiple cameras for uncalibrated visual servoing was studied. Switch images based on fusion were presented to get the continuous dynamic image Jacobian matrix among robots and distributed visual sensors. The designed fusion algorithm is suitable to have dynamically adjustable fusion weights, and the fusion structure was analyzed. Simulations and experiments without any knowledge of mobile robots and uncalibrated visual sensors show that the method has higher adaptability than the traditional instant switch control method. The method can enhance the system stability at the switching process.

A dual working mode mobile robot system based on visual guiding and visual servoing

A dual operational modes mobile robot system based on visual guiding and visual servo control is presented. This system consists of a mobile robot with a two-axis manipulator and a tele-operation station. In the visual guiding mode, for the robot works in an open loop visual servo control mode, the manipulating burden of the operator is reduced largely. In the visual servo mode the robot can locate the position of the target assigned by the operator and pick it up by its manipulator. With the help of the operator, the diffieuh problems of finding and handling a target in a complicated environment by the robot can be solved easily.

SPATIAL TRAJECTORY PREDICTION OF VISUAL SERVOING

Target tracking is one typical application of visual servoing technology. It is still a difficult task to track high speed target with current visual servo system. The improvement of visual servoing scheme is strongly required. A position-based visual servo parallel system is presented for tracking target with high speed. A local Frenet frame is assigned to the sampling point of spatial trajectory. Position estimation is formed by the differential features of intrinsic geometry, and orientation estimation is formed by homogenous transformation. The time spent for searching and processing can be greatly reduced by shifting the window according to features location prediction. The simulation results have demonstrated the ability of the system to track spatial moving object.

Visual servoing-based pneumatic hair transplantation mechanism for robotic FUE surgery

Hair transplantation surgery is an effective solution for hair loss problems,among which follicle unit extraction(FUE)surgery is more widely used and favored.At present,most hair transplantation surgeries still rely heavily on manual operation by doctors and very few hair transplantation robots with complex structures have been introduced.This paper proposes a pneumatic hair transplantation mechanism for FUE surgery,equipped with a camera,capable of automatically performing both hair implantation and extraction with airflow.This pneumatic method eliminates the complex needle structure,has the function of temporarily storing the follicles inside the needle,thus facilitating the automation of transferring follicles from extraction to implantation.Then a visual feedback system is proposed to accurately position the follicles during the transplantation.The experimental results show that average distance deviation between the actual and target positions is 0.6128 mm and the average deviation of hair implantation depth is 1.7176 mm,which verify the feasibility of the proposed system.

Image moments-based visual servoing control of bagged agricultural materials handling robot

Manual handling is less efficient and sometimes even hazardous to humans in many areas,for example,agriculture.Using robots in those areas not only avoids human contact with such dangerous agricultural materials but also improves working efficiency.The motion of a robot is controlled using a technique called visual servoing that uses feedback information extracted from a vision sensor.In this study,a visual servoing method was proposed based on learning features and image moments for 3D targets to solve the problem of image moment-based visual servoing.A Gaussian process regression model was used to map the relationship between the image moment invariants and the rotational angles around the X-and Y-axes of the camera frame(denoted asγandβ).To obtain maximal decoupled structure and minimal nonlinearities of the image Jacobian matrix,it was assumed two image moment features,which are linearly proportional toγandβ.In addition to the other four standardized image moment features,a 6-DOF image moment-based visual servoing controller for the agricultural material handling robot was designed.Using this method,the problem of visual servoing task failure due to the singularity of the Jacobian matrix was solved,and it also had a better convergence effect for the part of the target image beyond the field of view and large displacement visual servoing system.The proposed algorithm was validated by carrying out experiments tracking bagged flour in a six-degree-of-freedom robotic system.The final displacement positioning accuracy reached the millimeter level and the direction angle positioning accuracy reached the level of 0.1°.The method still has a certain convergence effect when the target image is beyond the field of view.The experimental results have been presented to show the adequate behavior of the presented approach in robot handling operations.It provides reference for the application of visual servoing technology in the field of agricultural robots and has important theoretical significance and practical value.

Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor:System,Algorithms,and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.

Calibration-free and model-independent method for high-DOF image-based visual servoing

This paper presents a novel method to improve the performance of high-DOF image base visual servoing （IBVS） with an uncalibrated camera. Firstly, analysis and comparison between point-based and moment-based features are carried out with respect to a 4-DOF positioning task. Then, an extended interaction matrix （IM） related to the digital image, and a Kalman filter （KF）-based estimation algorithm of the extended IM without calibration and IM model are proposed. Finally, the KF-based algorithm is extended to realize an approximation to decoupled control scheme. Experimental results conducted on an industrial robot show that our proposed methods can provide accurate estimation of IM, and achieve similar performance compared with traditional calibration-based method. Therefore, the proposed methods can be applied to any robot control system in variational environments, and can realize instant operation to planar object with complex and unknown shape at large displacement.

Adaptive tracking control for uncertain dynamic nonholonomic mobile robots based on visual servoing

The trajectory tracking control problem of dynamic nonholonomic wheeled mobile robots is considered via visual servoing feedback. A kinematic controller is firstly presented for the kinematic model, and then, an adaptive sliding mode controller is designed for the uncertain dynamic model in the presence of parametric uncertainties associated with the camera system. The proposed controller is robust not only to structured uncertainties such as mass variation but also to unstructured one such as disturbances. The asymptotic convergence of tracking errors to equilibrium point is rigorously proved by the Lyapunov method. Simulation results are provided to illustrate the performance of the control law.

The Exponential Stabilization of Uncertain Chained Form Systems of Mobile Robots Based on Visual Servoing

A system for a type of nonholonomic wheeled mobile robots equipped with an uncalibrated camera fixed to the ceiling is investigated.Based on the visual feedback and the state-input transformation,models of uncertain chained form systems are presented for the robot-camera systems.Then,new smooth time-varying feedback controllers are proposed to exponentially stabilize the uncertain chained system by using state-scaling and control theories for two cases.The exponential stabilities of the closed-loop uncertain systems are rigorously proved.Simulation results demonstrate the effectiveness of the proposed strategies.

Bionic tracking method by hand ＆ eye-vergence visual servoing

Animals rotate their eyes to gaze at the target prey, enhancing the ability of measuring the distance to the target precisely for catching it. These animals, visual tracking includes the triangular eye-vergence control and their body＇s motion control by visual servoing. The research aims to realize a bionic robot tracking perfor- mance, in which the body links moves together with eyes＇ view orientation. This paper proposed a hand ＆ eye-ver- gence dual control system which included two feedback loops： an outer loop for conventional visual servoing to direct a manipulator toward a target object and an inner loop for active motion control of binocular cameras to change the viewpoint along with the moving object to give an accurate and broad observation. This research also foused on how to compensate a fictional motion of the target seen by camera images in an eye-in-hand system, where the camera was fixed on the end-effector and moved together with the hand motion. A robust motion-feedfor- ward （MFF） recognition method is proposed to compensate the fictional motion of the target based on the manipula- tor＇s joint velocity, then the real motion of the target seen by camera images is extracted, which can improve the feedback image sensing unit to make the whole servoing system dynamically stable. The effectiveness of the pro- posed hand ＆ eye-vergence visual servoing method is shown by tracking experiments using a 6-DoF robot manipulator and a 3-DoF binocular vision system.

无人飞行器在不确定扰动下的动态目标跟踪

This study proposes an image-based visual servoing(IBVS)method based on a velocity observer for an unmanned aerial vehicle(UAV)for tracking a dynamic target in Global Positioning System(GPS)-denied environments.The proposed method derives the simplified and decoupled image dynamics of underactuated UAVs using a constructed virtual camera and then considers the uncertainties caused by the unpredictable rotations and velocities of the dynamic target.A novel image depth model that extends the IBVS method to track a rotating target with arbitrary orientations is proposed.The depth model ensures image feature accuracy and image trajectory smoothness in rotating target tracking.The relative velocities of the UAV and the dynamic target are estimated using the proposed velocity observer.Thanks to the velocity observer,translational velocity measurements are not required,and the control chatter caused by noise-containing measurements is mitigated.An integral-based filter is proposed to compensate for unpredictable environmental disturbances in order to improve the antidisturbance ability.The stability of the velocity observer and IBVS controller is analyzed using the Lyapunov method.Comparative simulations and multistage experiments are conducted to illustrate the tracking stability,anti-disturbance ability,and tracking robustness of the proposed method with a dynamic rotating target.

Manipulator tracking technology based on FSRUKF

Aiming at the shortcoming that the traditional industrial manipulator using off-line programming cannot change along with the change of external environment,the key technologies such as machine vision and manipulator control are studied,and a complete manipulator vision tracking system is designed.Firstly,Denavit-Hartenberg(D-H)parameters method is used to construct the model of the manipulator and analyze the forward and inverse kinematics equations of the manipulator.At the same time,a binocular camera is used to obtain the threedimensional position of the target.Secondly,in order to make the manipulator track the target more accurately,the fuzzy adaptive square root unscented Kalman filter(FSRUKF)is proposed to estimate the target state.Finally,the manipulator tracking system is built by using the position-based visual servo.The simulation experiments show that FSRUKF converges faster and with less error than the square root unscented Kalman filter(SRUKF),which meets the application requirements of the manipulator tracking system,and basically meets the application requirements of the manipulator tracking system in the practical experiments.