Robot Vision System for Coordinate Measurement of Feature Points on Large Scale Automobile Part

In this paper,we present a robot vision based system for coordinate measurement of feature points on large scale automobile parts.Our system consists of an industrial 6-DOF robot mounted with a CCD camera and a PC.The system controls the robot into the area of feature points.The images of measuring feature points are acquired by the camera mounted on the robot.3D positions of the feature points are obtained from a model based pose estimation that applies to the images.The measured positions of all feature points are then transformed to the reference coordinate of feature points whose positions are obtained from the coordinate measuring machine（CMM）.Finally,the point-to-point distances between the measured feature points and the reference feature points are calculated and reported.The results show that the root mean square error（RMSE） of measure values obtained by our system is less than 0.5 mm.Our system is adequate for automobile assembly and can perform faster than conventional methods.

Vision-based Stabilization of Nonholonomic Mobile Robots by Integrating Sliding-mode Control and Adaptive Approach

Vision-based pose stabilization of nonholonomic mobile robots has received extensive attention. At present, most of the solutions of the problem do not take the robot dynamics into account in the controller design, so that these controllers are difficult to realize satisfactory control in practical application. Besides, many of the approaches suffer from the initial speed and torque jump which are not practical in the real world. Considering the kinematics and dynamics, a two-stage visual controller for solving the stabilization problem of a mobile robot is presented, applying the integration of adaptive control, sliding-mode control, and neural dynamics. In the first stage, an adaptive kinematic stabilization controller utilized to generate the command of velocity is developed based on Lyapunov theory. In the second stage, adopting the sliding-mode control approach, a dynamic controller with a variable speed function used to reduce the chattering is designed, which is utilized to generate the command of torque to make the actual velocity of the mobile robot asymptotically reach the desired velocity. Furthermore, to handle the speed and torque jump problems, the neural dynamics model is integrated into the above mentioned controllers. The stability of the proposed control system is analyzed by using Lyapunov theory. Finally, the simulation of the control law is implemented in perturbed case, and the results show that the control scheme can solve the stabilization problem effectively. The proposed control law can solve the speed and torque jump problems, overcome external disturbances, and provide a new solution for the vision-based stabilization of the mobile robot.

Design of Robot Welding Seam Tracking System with Structured Light Vision

Robot welding is an important developing direction of welding automation and intelligentization, and automatic seam tracking technology is one of principal research domains. Nowadays, seam tracking system with structured light vision becomes a hot research. Structured light vision seam tracking products abroad are generally very expensive and can only be applied on special occasions. In China, the research of structured light vision seam tracking system is still just on the stage of experiments. A robot real-time seam tracking system with line structured light vision is designed. The hardware system is set up, a filtering method for line structure seam image is improved, and compared with common filtering, it has better effect and characteristic of real time. Two methods, fast template matching and fast Hough transform, to recognize the image coordinates of seam center are improved. Two new image recognition methods, structure element matching and comer detecting, are proposed. The comparison of seam image recognition shows that fast template matching and comer detecting are more precise and stable than the other two methods, and comer detecting is the best in real time. A simultaneous calibration for camera parameters and robot hand-eye is also proposed, and calculation shows that the calibration is effective and feasible. The robot seam tracking tests for linear and folded lap-joint are performed, which are based on the above four image recognition methods, and the results indicate that four image recognition methods are all applicable to real-time seam tracking, and the whole system sufficed for the requirements of real-time seam tracking. Automatic seam tracking with line structured light vision is feasible and has good versatility.

VISION-BASED RECOGNITION AND GUIDING OF INITIAL WELDING POSITION FOR ARC-WELDING ROBOT

A method is put forward to realize the recognition and guiding of initial welding position. The weld seams are marked with black lines, which simplify the computational complexity of image processing greatly. A two-time template matching method has been advanced to search for the target point, which is simple and has higher calculation speed. According to the depth computing principle with the special point matching using binocular stereovision, the initial welding position can be confirmed by calculating the middle point of the perpendicular line of two radials in the space. Taking the welding of propellant fuel container for example, good results are obtained with the algorithms. Finally, similar method for terminating welding position is also advanced.

Dead-reckoning/vision integrated navigation for mobile robot

A dead reckoning system and a vision navigation system are proposed for use in a new integrated system for robot navigation. Since the dead reckoning system uses a recurrence algorithm to determine the position, the position will be divergent in two horizontal directions with time increasing. In order to overcome this defect, a vision navigation system is used to periodically correct the dead reckoning system, and a kalman filter is used to estimate the errors of navigation and the unknown biases of sensors, and precise position and heading estimations are obtained by updating navigation errors and sensors’ biases. It is concluded from the simulation results that all the navigation parameters can be obtained through kalman filtering, and the integrated navigation system proposed for robot navigation can be used in an actual robot working in a laboratory. The measurement noise analysis shows that with the distance between beacon and robot increasing, the measurement noise will increase, and in order to achieve a proper estimation accuracy, the distance should not be too great.

Robot stereo vision calibration method with genetic algorithm and particle swarm optimization

Accurate stereo vision calibration is a preliminary step towards high-precision visual posi- tioning of robot. Combining with the characteristics of genetic algorithm （GA） and particle swarm optimization （PSO）, a three-stage calibration method based on hybrid intelligent optimization is pro- posed for nonlinear camera models in this paper. The motivation is to improve the accuracy of the calibration process. In this approach, the stereo vision calibration is considered as an optimization problem that can be solved by the GA and PSO. The initial linear values can be obtained in the frost stage. Then in the second stage, two cameras＇ parameters are optimized separately. Finally, the in- tegrated optimized calibration of two models is obtained in the third stage. Direct linear transforma- tion （DLT）, GA and PSO are individually used in three stages. It is shown that the results of every stage can correctly find near-optimal solution and it can be used to initialize the next stage. Simula- tion analysis and actual experimental results indicate that this calibration method works more accu- rate and robust in noisy environment compared with traditional calibration methods. The proposed method can fulfill the requirements of robot sophisticated visual operation.

Design of vision-based soccer robot using DSP

A new design of vision based soccer robot using the type TMS320F240 of DSPs for MiroSot series is presented. The DSP used enables cost effective control of DC motor, and features fewer external components, lower system cost and better performances than traditional microcontroller. The hardware architecture of robot is firstly presented in detail, and then the software design is briefly discussed. The control structure of decision making subsystem is illuminated also in this paper. The conclusion and prospect are given at last.

Robotic system for adding tundish-covering flux based on machine vision

Tundish-covering flux bags can be depalletized and moved in the steel casting region using industrial robots and monocular vision simultaneously.An industrial robot mounted with a flexible vacuum sucker was used as the executor.For a structured bag model,a visual scheme based on the support vector machine and the histogram of oriented gradients was adopted.The computer was trained using a number of sample bag images that relied on the feature recognition algorithm.Finally,the automatic stacking and moving of the flux bags were realized.

Learning-based force servoing control of a robot with vision in an unknown environment

A learning-based control approach is presented for force servoing of a robot with vision in an unknown environment. Firstly, mapping relationships between image features of the servoing object and the joint angles of the robot are derived and learned by a neural network. Secondly, a learning controller based on the neural network is designed for the robot to trace the object. Thirdly, a discrete time impedance control law is obtained for the force servoing of the robot, the on-line learning algorithms for three neural networks are developed to adjust the impedance parameters of the robot in the unknown environment. Lastly, wiping experiments are carried out by using a 6 DOF industrial robot with a CCD camera and a force/torque sensor in its end effector, and the experimental results confirm the effecti veness of the approach.

Hybrid Visual Servoing Control for Robotic Arc Welding Based on Structured Light Vision

A novel hybrid visual servoing control method based on structured light vision is pro-posed for robotic arc welding with a general six degrees of freedom robot. It consists of a positioncontrol inner-loop in Cartesian space and two outer-loops. One is position-based visual control inCartesian space for moving in the direction of weld seam, i.e., weld seam tracking, another is image-based visual control in image space for adjustment to eliminate the errors in the process of tracking.A new Jacobian matrix from image space of the feature point on structured light stripe to Cartesianspace is provided for dierential movement of the end-e?ector. The control system model is simplifiedand its stability is discussed. An experiment of arc welding protected by gas CO2 for verifying iswell conducted.

Planar Motion Tracking with an Uncalibrated Robot/Vision System

This paper discusses the coordination process for a robot gripper to approach a movingobject with feedback from an uncalibrated visual system. The dynamic of the whole system, includingtarget's random motion and the gripper's tracking motion, is bounded to a 2-D working plane. Acamera, whose relations with the robot system and the 2-D working plane are unknown to the robotcontroller, is fixed aside to observe the object and gripper positions continually. Thus the movementsof the robot gripper can be decided on the positions of the object observed in each visual samplingmoment. The coordination of the vision and robot system is to be shown independently from therelations between the robot and the vision system, which should always be calibrated a prior forthe control of traditional robot/vision coordination system. Simulations are provided to show theproperty of the proposed method.

A Novel Vision Localization Method of Automated Micro-Polishing Robot

Based on photogrammetry technology,a novel localization method of micro-polishing robot,which is restricted within certain working space,is presented in this paper.On the basis of pinhole camera model,a new mathematical model of vision localization of automated polishing robot is established.The vision localization is based on the distance-constraints of feature points.The method to solve the mathematical model is discussed.According to the characteristics of gray image,an adaptive method of automatic threshold selection based on connected components is presented.The center coordinate of the feature image point is resolved by bilinear interpolation gray square weighted algorithm.Finally,the mathematical model of testing system is verified by global localization test.The experimental results show that the vision localization system in working space has high precision.

Development of a Musical-based Interaction System for the Waseda Flutist Robot-Implementation of a Real-time Vision Interface Based on the Particle Filter Algorithm

The aim of this paper is to create an interface for human-robot interaction.Specifically,musical performance parameters(i.e.vibrato expression)of the Waseda Flutist Robot No.4 Refined IV(WF-4RIV)are to be manipulated.This research focused on enabling the WF-4RIV to interact with human players(musicians)in a natural way.In this paper,as the first approach,a vision processing algorithm,which is able to track the 3D-orientation and position of a musical instrument,was developed.In particular,the robot acquires image data through two cameras attached to its head.Using color histogram matching and a particle filter,the position of the musician's hands on the instrument are tracked.Analysis of this data determines orientation and location of the instrument.These parameters are mapped to manipulate the musical expression of the WF-4RIV,more specifically sound vibrato and volume values.The authors present preliminary experiments to determine if the robot may dynamically change musical parameters while interacting with a human player(i.e.vibrato etc.).From the experimental results,they may confirm the feasibility of the interaction during the performance,although further research must be carried out to consider the physical constraints of the flutist robot.

Enhancement of Process Capability for the Vision-Guided Robot

This study addresses a critical problem in the control of process capability as to the positioning accuracy of vision-guided robot. Depending on the calibration accuracy, the process capability varies widely, which renders the precise control of assembly tasks difficult. Furthermore, some vision sensors prohibit the programming access to rectify the lens distortion effects, which even complicates the problem. This study proposes a method of circumventing the lack of programming access by implementing the lens optical center alignment. Three different calibration methods are compared as to the process capability, and the proposed method shows a very good accuracy. The method can be easily adopted on the shop floor since it doesn’t require a complex setup and mathematical derivation process. Therefore, the practitioners can benefit from the proposed method, while maintaining a high level of precision in terms of robot positioning accuracy.

Research on stereo vision path-planning algorithms for mobile robots autonomous navigation

Using stereo vision for autonomous mobile robot path-planning is a hot technology.The environment mapping and path-planning algorithms were introduced,and they were applied in the autonomous mobile robot experiment platform.Through experiments in the robot platform,the effectiveness of these algorithms was verified.

Influence of Data Clouds Fusion From 3D RealTime Vision System on Robotic Group Dead Reckoning in Unknown Terrain

This paper proposes the solution of tasks set required for autonomous robotic group behavior optimization during the mission on a distributed area in a cluttered hazardous terrain.The navigation scheme uses the benefits of the original real-time technical vision system(TVS)based on a dynamic triangulation principle.The method uses TVS output data with fuzzy logic rules processing for resolution stabilization.Based on previous researches,the dynamic communication network model is modified to implement the propagation of information with a feedback method for more stable data exchange inside the robotic group.According to the comparative analysis of approximation methods,in this paper authors are proposing to use two-steps post-processing path planning aiming to get a smooth and energy-saving trajectory.The article provides a wide range of studies and computational experiment results for different scenarios for evaluation of common cloud point influence on robotic motion planning.

New self-calibration approach to space robots based on hand-eye vision

To overcome the influence of on-orbit extreme temperature environment on the tool pose(position and orientation) accuracy of a space robot,a new self-calibration method based on a measurement camera(hand-eye vision) attached to its end-effector was presented.Using the relative pose errors between the two adjacent calibration positions of the space robot,the cost function of the calibration was built,which was different from the conventional calibration method.The particle swarm optimization algorithm(PSO) was used to optimize the function to realize the geometrical parameter identification of the space robot.The above calibration method was carried out through self-calibration simulation of a six-DOF space robot whose end-effector was equipped with hand-eye vision.The results showed that after calibration there was a significant improvement of tool pose accuracy in a set of independent reference positions,which verified the feasibility of the method.At the same time,because it was unnecessary for this method to know the transformation matrix from the robot base to the calibration plate,it reduced the complexity of calibration model and shortened the error propagation chain,which benefited to improve the calibration accuracy.