Sliding Mode Variable Structure Control for Visual Servoing System

A visual servoing tracking controller is proposed based on the sliding mode control theory in order to achieve strong robustness against parameter variations and external disturbances. A sliding plane with time delay compensation is presented by the pre-estimate of states. To reduce the chattering of the sliding mode controller, a modified exponential reaching law and hyperbolic tangent function are applied to the design of visual controller and robot joint controller. Simulation results show that the visual servoing control scheme is robust and has good tracking performance.

SVM-based Identification and Un-calibrated Visual Servoing for Micro-manipulation

This paper presents an improved support vector machine （SVM） algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set＇s discernible matrix is proposed to identify and classify micro-targets. To avoid the complicated calibration for intrinsic parameters of camera, an improved Broyden＇s method is proposed to estimate the image Jacobian matrix which employs Chebyshev polynomial to construct a cost function to approximate the optimization value. Finally, a visual controller is designed for a robotic micromanipulation system. The experiment results of micro-parts assembly show that the proposed methods and algorithms are effective and feasible.

Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator

This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s sixdimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.

CONTROL SCHEMES FOR CMAC NEURAL NETWORK-BASED VISUAL SERVOING

In IBVS (image based visual servoing), the error signal in image space should be transformed into the control signal in the input space quickly. To avoid the iterative adjustment and complicated inverse solution of image Jacobian, CMAC (cerebellar model articulation controller) neural network is inserted into visual servo control loop to implement the nonlinear mapping. Two control schemes are used. Simulation results on two schemes are provided, which show a better tracking precision and stability can be achieved using scheme 2.

SPATIAL TRAJECTORY PREDICTION OF VISUAL SERVOING

Target tracking is one typical application of visual servoing technology. It is still a difficult task to track high speed target with current visual servo system. The improvement of visual servoing scheme is strongly required. A position-based visual servo parallel system is presented for tracking target with high speed. A local Frenet frame is assigned to the sampling point of spatial trajectory. Position estimation is formed by the differential features of intrinsic geometry, and orientation estimation is formed by homogenous transformation. The time spent for searching and processing can be greatly reduced by shifting the window according to features location prediction. The simulation results have demonstrated the ability of the system to track spatial moving object.

Image moments-based visual servoing control of bagged agricultural materials handling robot

Manual handling is less efficient and sometimes even hazardous to humans in many areas,for example,agriculture.Using robots in those areas not only avoids human contact with such dangerous agricultural materials but also improves working efficiency.The motion of a robot is controlled using a technique called visual servoing that uses feedback information extracted from a vision sensor.In this study,a visual servoing method was proposed based on learning features and image moments for 3D targets to solve the problem of image moment-based visual servoing.A Gaussian process regression model was used to map the relationship between the image moment invariants and the rotational angles around the X-and Y-axes of the camera frame(denoted asγandβ).To obtain maximal decoupled structure and minimal nonlinearities of the image Jacobian matrix,it was assumed two image moment features,which are linearly proportional toγandβ.In addition to the other four standardized image moment features,a 6-DOF image moment-based visual servoing controller for the agricultural material handling robot was designed.Using this method,the problem of visual servoing task failure due to the singularity of the Jacobian matrix was solved,and it also had a better convergence effect for the part of the target image beyond the field of view and large displacement visual servoing system.The proposed algorithm was validated by carrying out experiments tracking bagged flour in a six-degree-of-freedom robotic system.The final displacement positioning accuracy reached the millimeter level and the direction angle positioning accuracy reached the level of 0.1°.The method still has a certain convergence effect when the target image is beyond the field of view.The experimental results have been presented to show the adequate behavior of the presented approach in robot handling operations.It provides reference for the application of visual servoing technology in the field of agricultural robots and has important theoretical significance and practical value.

Visual servoing-based pneumatic hair transplantation mechanism for robotic FUE surgery

Hair transplantation surgery is an effective solution for hair loss problems,among which follicle unit extraction(FUE)surgery is more widely used and favored.At present,most hair transplantation surgeries still rely heavily on manual operation by doctors and very few hair transplantation robots with complex structures have been introduced.This paper proposes a pneumatic hair transplantation mechanism for FUE surgery,equipped with a camera,capable of automatically performing both hair implantation and extraction with airflow.This pneumatic method eliminates the complex needle structure,has the function of temporarily storing the follicles inside the needle,thus facilitating the automation of transferring follicles from extraction to implantation.Then a visual feedback system is proposed to accurately position the follicles during the transplantation.The experimental results show that average distance deviation between the actual and target positions is 0.6128 mm and the average deviation of hair implantation depth is 1.7176 mm,which verify the feasibility of the proposed system.

Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor:System,Algorithms,and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.

Calibration-free and model-independent method for high-DOF image-based visual servoing

This paper presents a novel method to improve the performance of high-DOF image base visual servoing （IBVS） with an uncalibrated camera. Firstly, analysis and comparison between point-based and moment-based features are carried out with respect to a 4-DOF positioning task. Then, an extended interaction matrix （IM） related to the digital image, and a Kalman filter （KF）-based estimation algorithm of the extended IM without calibration and IM model are proposed. Finally, the KF-based algorithm is extended to realize an approximation to decoupled control scheme. Experimental results conducted on an industrial robot show that our proposed methods can provide accurate estimation of IM, and achieve similar performance compared with traditional calibration-based method. Therefore, the proposed methods can be applied to any robot control system in variational environments, and can realize instant operation to planar object with complex and unknown shape at large displacement.

Adaptive tracking control for uncertain dynamic nonholonomic mobile robots based on visual servoing

The trajectory tracking control problem of dynamic nonholonomic wheeled mobile robots is considered via visual servoing feedback. A kinematic controller is firstly presented for the kinematic model, and then, an adaptive sliding mode controller is designed for the uncertain dynamic model in the presence of parametric uncertainties associated with the camera system. The proposed controller is robust not only to structured uncertainties such as mass variation but also to unstructured one such as disturbances. The asymptotic convergence of tracking errors to equilibrium point is rigorously proved by the Lyapunov method. Simulation results are provided to illustrate the performance of the control law.

Bionic tracking method by hand ＆ eye-vergence visual servoing

Animals rotate their eyes to gaze at the target prey, enhancing the ability of measuring the distance to the target precisely for catching it. These animals, visual tracking includes the triangular eye-vergence control and their body＇s motion control by visual servoing. The research aims to realize a bionic robot tracking perfor- mance, in which the body links moves together with eyes＇ view orientation. This paper proposed a hand ＆ eye-ver- gence dual control system which included two feedback loops： an outer loop for conventional visual servoing to direct a manipulator toward a target object and an inner loop for active motion control of binocular cameras to change the viewpoint along with the moving object to give an accurate and broad observation. This research also foused on how to compensate a fictional motion of the target seen by camera images in an eye-in-hand system, where the camera was fixed on the end-effector and moved together with the hand motion. A robust motion-feedfor- ward （MFF） recognition method is proposed to compensate the fictional motion of the target based on the manipula- tor＇s joint velocity, then the real motion of the target seen by camera images is extracted, which can improve the feedback image sensing unit to make the whole servoing system dynamically stable. The effectiveness of the pro- posed hand ＆ eye-vergence visual servoing method is shown by tracking experiments using a 6-DoF robot manipulator and a 3-DoF binocular vision system.

无人飞行器在不确定扰动下的动态目标跟踪

This study proposes an image-based visual servoing(IBVS)method based on a velocity observer for an unmanned aerial vehicle(UAV)for tracking a dynamic target in Global Positioning System(GPS)-denied environments.The proposed method derives the simplified and decoupled image dynamics of underactuated UAVs using a constructed virtual camera and then considers the uncertainties caused by the unpredictable rotations and velocities of the dynamic target.A novel image depth model that extends the IBVS method to track a rotating target with arbitrary orientations is proposed.The depth model ensures image feature accuracy and image trajectory smoothness in rotating target tracking.The relative velocities of the UAV and the dynamic target are estimated using the proposed velocity observer.Thanks to the velocity observer,translational velocity measurements are not required,and the control chatter caused by noise-containing measurements is mitigated.An integral-based filter is proposed to compensate for unpredictable environmental disturbances in order to improve the antidisturbance ability.The stability of the velocity observer and IBVS controller is analyzed using the Lyapunov method.Comparative simulations and multistage experiments are conducted to illustrate the tracking stability,anti-disturbance ability,and tracking robustness of the proposed method with a dynamic rotating target.

Manipulator tracking technology based on FSRUKF

Aiming at the shortcoming that the traditional industrial manipulator using off-line programming cannot change along with the change of external environment,the key technologies such as machine vision and manipulator control are studied,and a complete manipulator vision tracking system is designed.Firstly,Denavit-Hartenberg(D-H)parameters method is used to construct the model of the manipulator and analyze the forward and inverse kinematics equations of the manipulator.At the same time,a binocular camera is used to obtain the threedimensional position of the target.Secondly,in order to make the manipulator track the target more accurately,the fuzzy adaptive square root unscented Kalman filter(FSRUKF)is proposed to estimate the target state.Finally,the manipulator tracking system is built by using the position-based visual servo.The simulation experiments show that FSRUKF converges faster and with less error than the square root unscented Kalman filter(SRUKF),which meets the application requirements of the manipulator tracking system,and basically meets the application requirements of the manipulator tracking system in the practical experiments.

Visual Avoidance of Collision with Randomly Moving Obstacles through Approximate Reinforcement Learning

In this research work,a hierarchical controller has been designed for an autonomous navigation robot to avoid unexpected moving obstacles where the state and action spaces are continuous.The proposed scheme consists of two parts:1)a controller with a high-level approximate reinforcement learning(ARL)technique for choosing an optimal trajectory in autonomous navigation;and 2)a low-level,appearance-based visual servoing(ABVS)controller which controls and execute the motion of the robot.A novel approach for path planning and visual servoing has been proposed by the combined system framework.The characteristics of the on-board camera which is equipped on the robot is naturally suitable for conducting the reinforcement learning algorithm.Regarding the ARL controller,the computational overhead is quite low thanks to the fact that a knowledge of obstacle motion is not necessary.The developed scheme has been implemented and validated in a simulation system of obstacle avoidance.It is noted that findings of the proposed method are successfully verified by obtaining an optimal robotic plan motion strategy.

Design of Robotic Visual Servo Control Based on Neural Network and Genetic Algorithm

A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP Mgorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.

A method of initial welding position guiding for arc welding robot based on visual servo control

In order to solve the visual guiding task of initial welding position for arc welding robot, this paper presents a practice prone image based visual servo control strategy without calibration, and we perform validating experiments on a nine DOF arc welding robot system. Experimental results illustrate presented method has the function to fulfill the task of welding robot initial positioning with certain anti jamming ability. This method provides a basis for guiding welding gun to initial welding pose with real typical seam’s image properties to replace flag block properties, and is a significant exploit to realize visual guiding of initial welding position and seam tracing in robot welding system.

An improved self-calibration approach based on adaptive genetic algorithm for position-based visual servo

An improved self-calibrating algorithm for visual servo based on adaptive genetic algorithm is proposed in this paper. Our approach introduces an extension of Mendonca-Cipolla and G. Chesi＇s self-calibration for the positionbased visual servo technique which exploits the singular value property of the essential matrix. Specifically, a suitable dynamic online cost function is generated according to the property of the three singular values. The visual servo process is carried out simultaneous to the dynamic self-calibration, and then the cost function is minimized using the adaptive genetic algorithm instead of the gradient descent method in G. Chesi＇s approach. Moreover, this method overcomes the limitation that the initial parameters must be selected close to the true value, which is not constant in many cases. It is not necessary to know exactly the camera intrinsic parameters when using our approach, instead, coarse coding bounds of the five parameters are enough for the algorithm, which can be done once and for all off-line. Besides, this algorithm does not require knowledge of the 3D model of the object. Simulation experiments are carried out and the results demonstrate that the proposed approach provides a fast convergence speed and robustness against unpredictable perturbations of camera parameters, and it is an effective and efficient visual servo algorithm.

Dual sliding mode coordinated control of manipulator grasping system with visual assistance

The operation efficiency of the manipulator is placed in the primary position in automatic production. This paper proposes a coordinated control strategy for joint servo and visual servo to enable timely transfer and accurate gripping in the working area. Aiming at the issues of chattering and slow convergence of traditional sliding mode controller, a fast variable power reaching rate on the basis of the non-singular fast terminal sliding mode controller is proposed, which can effectively reduce the convergence time and chattering. For the purpose of addressing the problem that the traditional visual servo control method is sensitive to the environment, a visual servo controller based on integral sliding mode is proposed, to ensure the favorable positioning accuracy of the manipulator. Based on the two proposed controllers mentioned above, a coordinated control strategy is used to implement the control of the manipulator. Finally, the upper computer software is developed using the C# programming language to monitor the workstation. The feasibility of the above-mentioned method is verified through multiple simulations and experiments.

The VILMA intelligent vehicle:an architectural design for cooperative control between driver and automated system

Intelligent autonomous vehicles have received a great degree of attention in recent years. Although the technology required for these vehicles is relatively advanced, the challenge is firstly to ensure that drivers can understand the capabilities and limitations of such systems and secondly to design a system that can handle the interaction between the driver and the automated intelligent system. In this study, we describe an approach using different strategies for an autonomous system and a driver to drive a vehicle cooperatively. The proposed strategies are referred to as cooperative planning and control and determine when and how the path projected by the autonomous system can be changed safely by the driver to a path that he wishes to follow. The first phase of the project is described, covering the design and implementation of an autonomous test vehicle. Experiments are carried out with a driver to test the cooperative planning and control concepts proposed here.

A Control Method for SMA Robotic Actuators

Soft robots show remarkable benefits over conventional rigid robots due to their high energy density and other factors. We propose a circular soft robot integrating the control system, actuating it with several shape memory alloy (SMA) actuators. Our research methodology involved connecting the DC voltage supply and L298N module to provide uninterrupted power to the actuator and thence control the actuator via Arduino Uno MCU and TOF camera. We designed the controller to simultaneously complete the positioning and manipulation tasks. A novel method utilizing visual servo and closed-loop control algorithm was proposed and integrated into the controller. This method involves the implementation of multi-gait locomotion using SMA actuators. Additionally, the development of closed-loop dynamic controllers for a continuous soft robot is also evaluated. The proposed control model is designed and simulated on the MATLAB tool. To verify the efficiency of the proposed forward-feedback controller, simulations and experiments were conducted in the current study. A new control method using PID control based on the Kalman filtering algorithm and visual servo for the SMA actuator designed in this research is introduced. We conclude that applying spike excitation voltage would benefit the actuating performance. Overall, the experimental results demonstrated a promising future for the purposed control method.