Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor:System,Algorithms,and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.

Sliding Mode Variable Structure Control for Visual Servoing System

A visual servoing tracking controller is proposed based on the sliding mode control theory in order to achieve strong robustness against parameter variations and external disturbances. A sliding plane with time delay compensation is presented by the pre-estimate of states. To reduce the chattering of the sliding mode controller, a modified exponential reaching law and hyperbolic tangent function are applied to the design of visual controller and robot joint controller. Simulation results show that the visual servoing control scheme is robust and has good tracking performance.

Design of Robotic Visual Servo Control Based on Neural Network and Genetic Algorithm

A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP Mgorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.

SVM-based Identification and Un-calibrated Visual Servoing for Micro-manipulation

This paper presents an improved support vector machine （SVM） algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set＇s discernible matrix is proposed to identify and classify micro-targets. To avoid the complicated calibration for intrinsic parameters of camera, an improved Broyden＇s method is proposed to estimate the image Jacobian matrix which employs Chebyshev polynomial to construct a cost function to approximate the optimization value. Finally, a visual controller is designed for a robotic micromanipulation system. The experiment results of micro-parts assembly show that the proposed methods and algorithms are effective and feasible.

Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator

This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s sixdimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.

A method of initial welding position guiding for arc welding robot based on visual servo control

In order to solve the visual guiding task of initial welding position for arc welding robot, this paper presents a practice prone image based visual servo control strategy without calibration, and we perform validating experiments on a nine DOF arc welding robot system. Experimental results illustrate presented method has the function to fulfill the task of welding robot initial positioning with certain anti jamming ability. This method provides a basis for guiding welding gun to initial welding pose with real typical seam’s image properties to replace flag block properties, and is a significant exploit to realize visual guiding of initial welding position and seam tracing in robot welding system.

An improved self-calibration approach based on adaptive genetic algorithm for position-based visual servo

An improved self-calibrating algorithm for visual servo based on adaptive genetic algorithm is proposed in this paper. Our approach introduces an extension of Mendonca-Cipolla and G. Chesi＇s self-calibration for the positionbased visual servo technique which exploits the singular value property of the essential matrix. Specifically, a suitable dynamic online cost function is generated according to the property of the three singular values. The visual servo process is carried out simultaneous to the dynamic self-calibration, and then the cost function is minimized using the adaptive genetic algorithm instead of the gradient descent method in G. Chesi＇s approach. Moreover, this method overcomes the limitation that the initial parameters must be selected close to the true value, which is not constant in many cases. It is not necessary to know exactly the camera intrinsic parameters when using our approach, instead, coarse coding bounds of the five parameters are enough for the algorithm, which can be done once and for all off-line. Besides, this algorithm does not require knowledge of the 3D model of the object. Simulation experiments are carried out and the results demonstrate that the proposed approach provides a fast convergence speed and robustness against unpredictable perturbations of camera parameters, and it is an effective and efficient visual servo algorithm.

CONTROL SCHEMES FOR CMAC NEURAL NETWORK-BASED VISUAL SERVOING

In IBVS (image based visual servoing), the error signal in image space should be transformed into the control signal in the input space quickly. To avoid the iterative adjustment and complicated inverse solution of image Jacobian, CMAC (cerebellar model articulation controller) neural network is inserted into visual servo control loop to implement the nonlinear mapping. Two control schemes are used. Simulation results on two schemes are provided, which show a better tracking precision and stability can be achieved using scheme 2.

SPATIAL TRAJECTORY PREDICTION OF VISUAL SERVOING

Target tracking is one typical application of visual servoing technology. It is still a difficult task to track high speed target with current visual servo system. The improvement of visual servoing scheme is strongly required. A position-based visual servo parallel system is presented for tracking target with high speed. A local Frenet frame is assigned to the sampling point of spatial trajectory. Position estimation is formed by the differential features of intrinsic geometry, and orientation estimation is formed by homogenous transformation. The time spent for searching and processing can be greatly reduced by shifting the window according to features location prediction. The simulation results have demonstrated the ability of the system to track spatial moving object.

Dual sliding mode coordinated control of manipulator grasping system with visual assistance

The operation efficiency of the manipulator is placed in the primary position in automatic production. This paper proposes a coordinated control strategy for joint servo and visual servo to enable timely transfer and accurate gripping in the working area. Aiming at the issues of chattering and slow convergence of traditional sliding mode controller, a fast variable power reaching rate on the basis of the non-singular fast terminal sliding mode controller is proposed, which can effectively reduce the convergence time and chattering. For the purpose of addressing the problem that the traditional visual servo control method is sensitive to the environment, a visual servo controller based on integral sliding mode is proposed, to ensure the favorable positioning accuracy of the manipulator. Based on the two proposed controllers mentioned above, a coordinated control strategy is used to implement the control of the manipulator. Finally, the upper computer software is developed using the C# programming language to monitor the workstation. The feasibility of the above-mentioned method is verified through multiple simulations and experiments.

Manipulator tracking technology based on FSRUKF

Aiming at the shortcoming that the traditional industrial manipulator using off-line programming cannot change along with the change of external environment,the key technologies such as machine vision and manipulator control are studied,and a complete manipulator vision tracking system is designed.Firstly,Denavit-Hartenberg(D-H)parameters method is used to construct the model of the manipulator and analyze the forward and inverse kinematics equations of the manipulator.At the same time,a binocular camera is used to obtain the threedimensional position of the target.Secondly,in order to make the manipulator track the target more accurately,the fuzzy adaptive square root unscented Kalman filter(FSRUKF)is proposed to estimate the target state.Finally,the manipulator tracking system is built by using the position-based visual servo.The simulation experiments show that FSRUKF converges faster and with less error than the square root unscented Kalman filter(SRUKF),which meets the application requirements of the manipulator tracking system,and basically meets the application requirements of the manipulator tracking system in the practical experiments.

无人飞行器在不确定扰动下的动态目标跟踪

This study proposes an image-based visual servoing(IBVS)method based on a velocity observer for an unmanned aerial vehicle(UAV)for tracking a dynamic target in Global Positioning System(GPS)-denied environments.The proposed method derives the simplified and decoupled image dynamics of underactuated UAVs using a constructed virtual camera and then considers the uncertainties caused by the unpredictable rotations and velocities of the dynamic target.A novel image depth model that extends the IBVS method to track a rotating target with arbitrary orientations is proposed.The depth model ensures image feature accuracy and image trajectory smoothness in rotating target tracking.The relative velocities of the UAV and the dynamic target are estimated using the proposed velocity observer.Thanks to the velocity observer,translational velocity measurements are not required,and the control chatter caused by noise-containing measurements is mitigated.An integral-based filter is proposed to compensate for unpredictable environmental disturbances in order to improve the antidisturbance ability.The stability of the velocity observer and IBVS controller is analyzed using the Lyapunov method.Comparative simulations and multistage experiments are conducted to illustrate the tracking stability,anti-disturbance ability,and tracking robustness of the proposed method with a dynamic rotating target.

基于视觉伺服的水下机器人导引技术

自主水下载具(AUV)需要通过停靠入站进行休眠、电池充电、传输数据等任务的需求。AUV的导引技术对延长AUV的工作时长、扩大作业范围有重要意义。在入站过程的最后阶段,AUV需要在容许范围内保持自身与对象物的相对位姿(包括位置和方向),进而完成对接。因此,确保传感器单元和控制系统在真实的水下环境中对各种干扰具有高度精确性和鲁棒性至关重要。针对此问题,提出了一种基于视觉伺服的AUV的导引技术。首先,该技术通过模型匹配认识法确定机器人与指定标的物(对象物)之间的相对位姿来确定自己的方位。其次,通过遗传算法(GA)以完成实时的位姿认识,抵消水流、图像采集干扰带来的视觉伺服干扰。采用此方法的水下机器人控制系统分别在与东京大学合作的模拟深海水下无光环境实验,以及真实海洋环境中完成了实验验证。实验结果证明,该系统具有对低照度、高浊度引起的图像劣化以及水流引起的控制干扰具备抗干扰性和鲁棒性。

The VILMA intelligent vehicle:an architectural design for cooperative control between driver and automated system

Intelligent autonomous vehicles have received a great degree of attention in recent years. Although the technology required for these vehicles is relatively advanced, the challenge is firstly to ensure that drivers can understand the capabilities and limitations of such systems and secondly to design a system that can handle the interaction between the driver and the automated intelligent system. In this study, we describe an approach using different strategies for an autonomous system and a driver to drive a vehicle cooperatively. The proposed strategies are referred to as cooperative planning and control and determine when and how the path projected by the autonomous system can be changed safely by the driver to a path that he wishes to follow. The first phase of the project is described, covering the design and implementation of an autonomous test vehicle. Experiments are carried out with a driver to test the cooperative planning and control concepts proposed here.

机器人自动涂胶工艺质量与效率提升的应用研究

焊装车间的汽车零部件涂胶工艺在设计上采用传统的方式,要满足焊装车间多种产品、不同胶型、不同胶条直径同时作业的需求,在投资上费用较高且占地空间比较大。文章介绍了一种基于机器人的单泵多伺服定量机(胶枪)涂胶系统,并结合视觉检测技术在汽车零部件涂胶生产上的应用,从事前控制和事后监控两个方面,提高汽车制造生产中的机器人自动涂胶工艺质量与效率。通过“主控制系统”对单泵以及多个伺服定量机(胶枪)进行控制,能够基于单个涂胶泵同时对应多种产品、不同胶型直径的涂胶需求,调整机器人自动涂胶作业程序。同时能够有效提高焊装工序的汽车零部件涂胶设备的利用率,降低资金投入,有效确保涂胶质量。

Visual Avoidance of Collision with Randomly Moving Obstacles through Approximate Reinforcement Learning

In this research work,a hierarchical controller has been designed for an autonomous navigation robot to avoid unexpected moving obstacles where the state and action spaces are continuous.The proposed scheme consists of two parts:1)a controller with a high-level approximate reinforcement learning(ARL)technique for choosing an optimal trajectory in autonomous navigation;and 2)a low-level,appearance-based visual servoing(ABVS)controller which controls and execute the motion of the robot.A novel approach for path planning and visual servoing has been proposed by the combined system framework.The characteristics of the on-board camera which is equipped on the robot is naturally suitable for conducting the reinforcement learning algorithm.Regarding the ARL controller,the computational overhead is quite low thanks to the fact that a knowledge of obstacle motion is not necessary.The developed scheme has been implemented and validated in a simulation system of obstacle avoidance.It is noted that findings of the proposed method are successfully verified by obtaining an optimal robotic plan motion strategy.

Research on simulation system of virtual three-phase stepping motor

To change the current phenomenon of ＂Being an armchair strategist＂ in the research of a stepping motor, the virtual simu- lation system of stepping motor was developed using virtual reality technique. The basic principle of stepping motor was expounded. The development strategy of the simulation software is introduced. Some key problems and their solutions, including coordinate transformation of rotor and order of on-off electricity for stator, were put forward in system development. Finally, there are three virtual simulation examples of the on-off electricity way in this paper. This work provides a valuable auxiliary tool for the study of the stepping motor.

基于语义分割视觉伺服的种苗自动夹取系统设计

现代植物组织培养是一项耗时费力的工作,工作强度大,工作内容单调,为了减少劳动力成本并提高产量,设计了一种基于语义分割视觉伺服的种苗自动夹取系统并进行了测试。首先,提出了一种基于DP-BiSeNetV2语义分割算法的视觉定位方法,确定了根上合适的夹取点;之后,设计开发并测试了一种适合于实际工作环境的夹取装置;最后,将视觉定位算法与机器人夹取装置集成,构建了一个种苗自动夹取系统。在实验环节,使用蝴蝶兰种苗数据集进行了测试,在语义分割实验中,DP-BiSeNetV2模型的平均交并比为63.51%,像素准确度为98.25%;在夹取实验中,夹取成功率为81.7%。实验结果表明,该自动夹取系统具有很高的潜力,可以满足植物组织培养生产线的移植需求。

基于BP神经网络-模糊控制的机器人无标定视觉伺服技术

针对机器人无标定视觉伺服技术中图像雅可比矩阵在线估计存在计算复杂的问题,提出了一种结合反向传播(back propagation,BP)神经网络和模糊控制策略的机器人控制技术。以多自由度智能调节系统为例,提出其视觉伺服控制架构,根据工业场景数据集训练BP神经网络,采用本文所提算法进行法兰对中实验,帮助解决核电站蒸汽发生器人孔螺栓咬死问题。在方法层面,首先,利用BP神经网络建立图像特征信息与机器人多自由度运动之间的映射关系,之后,提出模糊控制方法根据图像特征偏差进行机器人位姿的精确调整。实验结果表明,本文算法能够有效应用于无标定视觉伺服控制,最终法兰平均对中误差在±1 mm内,平均耗时43 s,满足应用需求,具有较高的工作效率。

基于几何特征的铆接件精确装配视觉伺服控制方法

针对目前铆接件机器人装配多采用示教再现方式实现,主要应用于固定工况的问题,为满足铆钉数量多、工况多变的铆接件装配需求,提出一种基于几何特征的铆接件精确装配视觉伺服控制方法。首先,通过将相机安装在机器人末端执行器平台,搭建基于机器人的铆接件自主对准的单目视觉系统,完成相机、机器人末端与基坐标系标定;其次,选取铆钉平面初始图像中的4个铆钉圆心作为特征点,以特征点之间的连线构造直线特征,利用直线特征与相机旋转运动间的交互矩阵设计基于线特征的铆接件姿态控制律,使得铆钉与铆钉孔轴线平行,从而保证装配过程中铆接件的相互平行;再次,选取铆钉平面初始图像中的4个铆钉圆心作为特征点,1个铆钉的轮廓作为圆面积特征,利用点特征与相机旋转运动的交互矩阵设计铆接件的姿态补偿项,进而利用点特征和圆面积特征与相机平移运动间的交互矩阵设计与姿态控制律相解耦的位置控制律,实现铆钉与相应铆钉孔轴线的重合。最后,采用六自由度机器人进行铆接件自主对准装配仿真。结果表明,以铆接件内部铆钉固有的图像特征作为控制输入,所设计的控制器能够实现铆接件的自主高精度对准。方法对于工业生产中多点对准装配的工况具有广泛的应用价值。