Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor:System,Algorithms,and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.

Sliding Mode Variable Structure Control for Visual Servoing System

A visual servoing tracking controller is proposed based on the sliding mode control theory in order to achieve strong robustness against parameter variations and external disturbances. A sliding plane with time delay compensation is presented by the pre-estimate of states. To reduce the chattering of the sliding mode controller, a modified exponential reaching law and hyperbolic tangent function are applied to the design of visual controller and robot joint controller. Simulation results show that the visual servoing control scheme is robust and has good tracking performance.

A dual working mode mobile robot system based on visual guiding and visual servoing

A dual operational modes mobile robot system based on visual guiding and visual servo control is presented. This system consists of a mobile robot with a two-axis manipulator and a tele-operation station. In the visual guiding mode, for the robot works in an open loop visual servo control mode, the manipulating burden of the operator is reduced largely. In the visual servo mode the robot can locate the position of the target assigned by the operator and pick it up by its manipulator. With the help of the operator, the diffieuh problems of finding and handling a target in a complicated environment by the robot can be solved easily.

Study on algorithm of dynamic uncalibrated eye-in-hand visual servoing system

Currently, most visual servoing system must be calibrated, while it is impossible to calibrate cameras and robot models precisely in industrial practice, so a novel dynamic uncalibrated eye-in-hand visual servoing system of tracking a moving target is proposed. The method does not require calibration of camera and robot kinematic models. Vision guided algorithm for tracking dynamic image is developed through minimizing nonlinear objective function. For the large residual has not been approximated in dynamic environment and the change of composite image Jacobian with time increment has not been computed in visual servoing system now,large residuals are dynamic approximated and the change of composite image Jacobian at each iterative step is computed. Simulation results demonstrate the validity of these approaches.

Design of Robotic Visual Servo Control Based on Neural Network and Genetic Algorithm

A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP Mgorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.

SVM-based Identification and Un-calibrated Visual Servoing for Micro-manipulation

This paper presents an improved support vector machine （SVM） algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set＇s discernible matrix is proposed to identify and classify micro-targets. To avoid the complicated calibration for intrinsic parameters of camera, an improved Broyden＇s method is proposed to estimate the image Jacobian matrix which employs Chebyshev polynomial to construct a cost function to approximate the optimization value. Finally, a visual controller is designed for a robotic micromanipulation system. The experiment results of micro-parts assembly show that the proposed methods and algorithms are effective and feasible.

Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator

This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s sixdimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.

A method of initial welding position guiding for arc welding robot based on visual servo control

In order to solve the visual guiding task of initial welding position for arc welding robot, this paper presents a practice prone image based visual servo control strategy without calibration, and we perform validating experiments on a nine DOF arc welding robot system. Experimental results illustrate presented method has the function to fulfill the task of welding robot initial positioning with certain anti jamming ability. This method provides a basis for guiding welding gun to initial welding pose with real typical seam’s image properties to replace flag block properties, and is a significant exploit to realize visual guiding of initial welding position and seam tracing in robot welding system.

Robust visual servoing based Chinese calligraphy on a humanoid robot

A robust visual servoing system is investigated on a humanoid robot which grasps a brush in Chinese calligraphy task.The system is implemented based on uncalibrated visual servoing controller utilizing Kalman-Bucy filter,with the help of an object detector by continuously adaptive MeanShift(CAMShift) algorithm.Under this control scheme,a humanoid robot can satisfactorily grasp a brush without system modeling.The proposed method is shown to be robust and effective through a Chinese calligraphy task on a NAO robot.

An improved self-calibration approach based on adaptive genetic algorithm for position-based visual servo

An improved self-calibrating algorithm for visual servo based on adaptive genetic algorithm is proposed in this paper. Our approach introduces an extension of Mendonca-Cipolla and G. Chesi＇s self-calibration for the positionbased visual servo technique which exploits the singular value property of the essential matrix. Specifically, a suitable dynamic online cost function is generated according to the property of the three singular values. The visual servo process is carried out simultaneous to the dynamic self-calibration, and then the cost function is minimized using the adaptive genetic algorithm instead of the gradient descent method in G. Chesi＇s approach. Moreover, this method overcomes the limitation that the initial parameters must be selected close to the true value, which is not constant in many cases. It is not necessary to know exactly the camera intrinsic parameters when using our approach, instead, coarse coding bounds of the five parameters are enough for the algorithm, which can be done once and for all off-line. Besides, this algorithm does not require knowledge of the 3D model of the object. Simulation experiments are carried out and the results demonstrate that the proposed approach provides a fast convergence speed and robustness against unpredictable perturbations of camera parameters, and it is an effective and efficient visual servo algorithm.

CONTROL SCHEMES FOR CMAC NEURAL NETWORK-BASED VISUAL SERVOING

In IBVS (image based visual servoing), the error signal in image space should be transformed into the control signal in the input space quickly. To avoid the iterative adjustment and complicated inverse solution of image Jacobian, CMAC (cerebellar model articulation controller) neural network is inserted into visual servo control loop to implement the nonlinear mapping. Two control schemes are used. Simulation results on two schemes are provided, which show a better tracking precision and stability can be achieved using scheme 2.

Path Planning of Planar Kinked Line Seam by Visual Servoing for Robotic Welding

Welding path planning can substitute for the manual teaching process of the robot and can promote the autonomous level of the robotic welding. A path planning method by visual servoing was presented, in which the optimal angle of charge-coupled device (CCD) camera was also planned. Aiming at planning two forms of kinked line seams, obtuse angle seam and right angle seam, a practicable solution was put forward. In this solution, the intersection of two adjacent straight segments is detected in each local seam image, and if intersection is found, the seam errors are calculated using the next straight segment. The experimental results show that kinked line seam can be well planned using this solution.

A Stable Switch Method Based on Fusion in Uncalibrated Visual Servoing

Stable switch control between multiple cameras for uncalibrated visual servoing was studied. Switch images based on fusion were presented to get the continuous dynamic image Jacobian matrix among robots and distributed visual sensors. The designed fusion algorithm is suitable to have dynamically adjustable fusion weights, and the fusion structure was analyzed. Simulations and experiments without any knowledge of mobile robots and uncalibrated visual sensors show that the method has higher adaptability than the traditional instant switch control method. The method can enhance the system stability at the switching process.

Uncalibrated visual servoing design for competitive networked robots

In this paper,a method with an eye-in-hand configuration is developed to hit targets during visual tracking for the TLS(Tele-Light Saber) game.It is not necessary to calibrate camera parameters and predict the trajectory of the moving object.Firstly,the expression of the image Jacobian matrix for the eye-in-hand configuration is proposed,and then an update law is designed to estimate the image Jacobian online.Furthermore,a control scheme is presented and the Lyapunov method is employed to prove asymptotic convergence of image errors.No assumption for the moving objects is needed.Finally,both simulation and experimental results are shown to support the approach in this paper.

SPATIAL TRAJECTORY PREDICTION OF VISUAL SERVOING

Target tracking is one typical application of visual servoing technology. It is still a difficult task to track high speed target with current visual servo system. The improvement of visual servoing scheme is strongly required. A position-based visual servo parallel system is presented for tracking target with high speed. A local Frenet frame is assigned to the sampling point of spatial trajectory. Position estimation is formed by the differential features of intrinsic geometry, and orientation estimation is formed by homogenous transformation. The time spent for searching and processing can be greatly reduced by shifting the window according to features location prediction. The simulation results have demonstrated the ability of the system to track spatial moving object.

基于无人系统的智能视觉控制算法研究

无人系统应用范围的急剧扩大,使得视觉感知环境愈加复杂多变,致使传统视觉控制算法难以有效控制视觉传感器获取精准的视觉感知图像,从而影响无人系统的稳定运行,故提出基于无人系统的智能视觉控制算法研究。应用Gamma曲线非线性变换无人系统视觉感知图像灰度值,再应用灰度世界法来增强图像的对比度。以处理后的图像为基础,计算其图像矩,即空间矩、中心矩和归一化中心矩,以描述图像的全局和局部特性。根据得到的无人系统视觉感知信息,搭建智能视觉控制框架。获取期望图像特征矩阵,提取当前时刻图像特征矩阵,通过基于改进萤火虫算法的极限学习机对摄像机转角进行非线性映射,从而获取智能视觉控制定律,以此消除视觉感知图像误差,实现智能视觉的有效控制。实验结果显示:在不同实验组别背景下,应用提出算法获得的视觉控制平均时间最小值达到了1 s,视觉控制平均误差最小值达到了0.12%,充分证实了提出算法的应用性能更佳。

Dual sliding mode coordinated control of manipulator grasping system with visual assistance

The operation efficiency of the manipulator is placed in the primary position in automatic production. This paper proposes a coordinated control strategy for joint servo and visual servo to enable timely transfer and accurate gripping in the working area. Aiming at the issues of chattering and slow convergence of traditional sliding mode controller, a fast variable power reaching rate on the basis of the non-singular fast terminal sliding mode controller is proposed, which can effectively reduce the convergence time and chattering. For the purpose of addressing the problem that the traditional visual servo control method is sensitive to the environment, a visual servo controller based on integral sliding mode is proposed, to ensure the favorable positioning accuracy of the manipulator. Based on the two proposed controllers mentioned above, a coordinated control strategy is used to implement the control of the manipulator. Finally, the upper computer software is developed using the C# programming language to monitor the workstation. The feasibility of the above-mentioned method is verified through multiple simulations and experiments.

Manipulator tracking technology based on FSRUKF

Aiming at the shortcoming that the traditional industrial manipulator using off-line programming cannot change along with the change of external environment,the key technologies such as machine vision and manipulator control are studied,and a complete manipulator vision tracking system is designed.Firstly,Denavit-Hartenberg(D-H)parameters method is used to construct the model of the manipulator and analyze the forward and inverse kinematics equations of the manipulator.At the same time,a binocular camera is used to obtain the threedimensional position of the target.Secondly,in order to make the manipulator track the target more accurately,the fuzzy adaptive square root unscented Kalman filter(FSRUKF)is proposed to estimate the target state.Finally,the manipulator tracking system is built by using the position-based visual servo.The simulation experiments show that FSRUKF converges faster and with less error than the square root unscented Kalman filter(SRUKF),which meets the application requirements of the manipulator tracking system,and basically meets the application requirements of the manipulator tracking system in the practical experiments.

无人飞行器在不确定扰动下的动态目标跟踪

This study proposes an image-based visual servoing(IBVS)method based on a velocity observer for an unmanned aerial vehicle(UAV)for tracking a dynamic target in Global Positioning System(GPS)-denied environments.The proposed method derives the simplified and decoupled image dynamics of underactuated UAVs using a constructed virtual camera and then considers the uncertainties caused by the unpredictable rotations and velocities of the dynamic target.A novel image depth model that extends the IBVS method to track a rotating target with arbitrary orientations is proposed.The depth model ensures image feature accuracy and image trajectory smoothness in rotating target tracking.The relative velocities of the UAV and the dynamic target are estimated using the proposed velocity observer.Thanks to the velocity observer,translational velocity measurements are not required,and the control chatter caused by noise-containing measurements is mitigated.An integral-based filter is proposed to compensate for unpredictable environmental disturbances in order to improve the antidisturbance ability.The stability of the velocity observer and IBVS controller is analyzed using the Lyapunov method.Comparative simulations and multistage experiments are conducted to illustrate the tracking stability,anti-disturbance ability,and tracking robustness of the proposed method with a dynamic rotating target.

基于视觉伺服的水下机器人导引技术

自主水下载具(AUV)需要通过停靠入站进行休眠、电池充电、传输数据等任务的需求。AUV的导引技术对延长AUV的工作时长、扩大作业范围有重要意义。在入站过程的最后阶段,AUV需要在容许范围内保持自身与对象物的相对位姿(包括位置和方向),进而完成对接。因此,确保传感器单元和控制系统在真实的水下环境中对各种干扰具有高度精确性和鲁棒性至关重要。针对此问题,提出了一种基于视觉伺服的AUV的导引技术。首先,该技术通过模型匹配认识法确定机器人与指定标的物(对象物)之间的相对位姿来确定自己的方位。其次,通过遗传算法(GA)以完成实时的位姿认识,抵消水流、图像采集干扰带来的视觉伺服干扰。采用此方法的水下机器人控制系统分别在与东京大学合作的模拟深海水下无光环境实验,以及真实海洋环境中完成了实验验证。实验结果证明,该系统具有对低照度、高浊度引起的图像劣化以及水流引起的控制干扰具备抗干扰性和鲁棒性。