Design of Robot Welding Seam Tracking System with Structured Light Vision

Robot welding is an important developing direction of welding automation and intelligentization, and automatic seam tracking technology is one of principal research domains. Nowadays, seam tracking system with structured light vision becomes a hot research. Structured light vision seam tracking products abroad are generally very expensive and can only be applied on special occasions. In China, the research of structured light vision seam tracking system is still just on the stage of experiments. A robot real-time seam tracking system with line structured light vision is designed. The hardware system is set up, a filtering method for line structure seam image is improved, and compared with common filtering, it has better effect and characteristic of real time. Two methods, fast template matching and fast Hough transform, to recognize the image coordinates of seam center are improved. Two new image recognition methods, structure element matching and comer detecting, are proposed. The comparison of seam image recognition shows that fast template matching and comer detecting are more precise and stable than the other two methods, and comer detecting is the best in real time. A simultaneous calibration for camera parameters and robot hand-eye is also proposed, and calculation shows that the calibration is effective and feasible. The robot seam tracking tests for linear and folded lap-joint are performed, which are based on the above four image recognition methods, and the results indicate that four image recognition methods are all applicable to real-time seam tracking, and the whole system sufficed for the requirements of real-time seam tracking. Automatic seam tracking with line structured light vision is feasible and has good versatility.

Seam Tracking and Visual Control for Robotic Arc Welding Based on Structured Light Stereovision

A real-time arc welding robot visual control system based on a local network with a multi-level hierarchy is developed in this paper. It consists of an intelligence and human-machine interface level, a motion planning level, a motion control level and a servo control level. The last three levels form a local real-time open robot controller, which realizes motion planning and motion control of a robot. A camera calibration method based on the relative movement of the end-effector connected to a robot is proposed and a method for tracking weld seam based on the structured light stereovision is provided. Combining the parameters of the cameras and laser plane, three groups of position values in Cartesian space are obtained for each feature point in a stripe projected on the weld seam. The accurate three-dimensional position of the edge points in the weld seam can be calculated from the obtained parameters with an information fusion algorithm. By calculating the weld seam parameter from position and image data, the movement parameters of the robot used for tracking can be determined. A swing welding experiment of type V groove weld is successfully conducted, the results of which show that the system has high resolution seam tracking in real-time, and works stably and efficiently.

Seam Tracking Technology for Hyperbaric Underwater Welding

Automatic weld seam tracking technology to be used in hyperbaric underwater damaged pipeline repair welding is much more important, because of poor bevel preparation and severe working condition. A weld seam tracking system based on digital signal processing（DSP） passive light weld image processing technology has been established. A convenient charge coupled device（CCD） camera system was used in the high pressure environment with the help of an aperture and focus altering mechanism to guarantee overall image visibility in the scope of pressure below 0.7 MPa. The system can be used in the hyperbaric environment to pick up the real welding image of both the welding arc and the welding pool. The newly developed DSP technology was adopted to achieve the goal of system real time characteristics. An effective weld groove edge recognition technique including narrow interesting window opening, middle value wave filtering, Sobel operator weld edge detecting and edge searching in a defined narrow area was proposed to remove the guide error and system accuracy was ensured. The results of tracking simulation and real tracking application with arc striking have proved the validity and the accuracy of the mentioned system and the image processing method.

Seam tracking control for mobile welding robot based on vision sensor

To solve the seam tracking problem of mobile welding robot,a new controller based on the dynamics of mobile welding robot was designed using the method of backstepping kinematics into dynamics.A self-turning fuzzy controller and a fuzzy-Gaussian neural network(FGNN) controller were designed to complete coordinately controlling of cross-slider and wheels.The fuzzy-neural control algorithm was described by applying the Gaussian function and back propagation(BP) learning rule was used to tune the membership function in real time by applying the FGNN controller.To make the tracking more quickly and smoothly,the neural network controller based on dynamic model was designed,which utilized self-learning and self-adaptive ability of the neural network to deal with the partial uncertainty and the disturbances of the parameters of the robot dynamic model and real-time compensate the dynamics coupling.The results show that the selected control input torques make the system globally and asymptotically stable based on the Lyapunov function selected out;the accuracy of the proposed controller tracing is within ±0.4 mm and can satisfy the requirements of practical welding project.

Study of image processing for V-shape groove and robotic weld seam tracking based on laser vision

Single-stripe laser was applied to acquire V-shape groove contour information. Most of arc light and splash noise was removed and stripe laser image was kept by wavelet transform. Half-threshold algorithm was used for image segmentation and stripe laser image was gotten after refining. Weld seam center position was identified and extracted by extreme curvature corner detection method. The location of torch was detected to accord the frequency of computer program with robot program and serial communication program. The tracking experiments of sidelong, reflex and curve weld line show that the system can meet the demand of the tracking precision under normal welding conditions.

The characteristics of welding currents in a rotating arc leadtandem GMAW process and the weld-seam tracking

Tandem gas metal arc welding （ T-GMAW） process shows a high deposition rate that up to three times o f the single electrode GMAW, so the welding speed could be significantly increased in this process. However, the majority o f this process applications are based on the pre-programmed robotic welding, which does not allow them to track the seam real-time during welding. Rotating arc sensor, sensing the seam position by detecting the changing of welding currents, has been widely adopted in the automatic robot welding process. It is proposed in this paper to integrate the rotating arc sensor with a trailing torch to develop a new approach of rotating arc lead tandem gas metal arc welding （RLT-GMAW） process. The characteristics of the welding currents in the proposed new welding process were firstly studied, and then a self-turning fuzzy control seam tracking strategy was developed for the mobile robot automatic welding. The experimental results showed that the proposed RLT-GMAW process had an excellent seam tracking performance and high welding deposition rate. Even if there were some electromagnetic interactions between the two arcs, the deviation of the welding seam could also be reflected by the fluctuation of the welding currents on the leading arc once the correct welding parameters were selected. Based on the detected deviation, the welding tracking experiments showed that the proposed self-turning fuzzy controller had a good performance for the RLT-GMAW process seam tracking.

Study on contactless ultrasonic sensor and seam tracking

The sensitivity of contactless ultrasonic sensor is improved from 3.3% to 26% through acoustic and electric matching technique in this paper.Along dual crest lines auto seam tracking is firstly realized with contactless ultrasonic sensing.The focused acoustic lens is accurately designed, and a sonic beam with 0.5 mm in diameter is achieved. By means of software and hardware technique the accuracy of seam tracking with ultrasonic sensing is 0.5 mm in lateral direction and 0.2 mm in longitudinal direction respectively.

Novel Real-Time Seam Tracking Algorithm Based on Vector Angle and Least Square Method

Real-time seam tracking can improve welding quality and enhance welding efficiency during the welding process in automobile manufacturing.However,the teaching-playing welding process,an off-line seam tracking method,is still dominant in automobile industry,which is less flexible when welding objects or situation change.A novel real-time algorithm consisting of seam detection and generation is proposed to track seam.Using captured 3D points,space vectors were created between two adjacent points along each laser line and then a vector angle based algorithm was developed to detect target points on the seam.Least square method was used to fit target points to a welding trajectory for seam tracking.Furthermore,the real-time seam tracking process was simulated in MATLAB/Simulink.The trend of joint angles vs.time was logged and a comparison between the off-line and the proposed seam tracking algorithm was conducted.Results show that the proposed real-time seam tracking algorithm can work in a real-time scenario and have high accuracy in welding point positioning.

A machine vision approach to seam tracking in real-time in PAW of large-diameter stainless steel tube

Manual monitoring and seam tracking through watching weld pool images in real-time, by naked eyes or by industrial TV, are experience-depended, subjective, labor intensive, and sometimes biased. So it is necessary to realize the automation of computer-aided seam tracking. A PAW (plasma arc welding) seam tracking system was developed, which senses the molten pool and the seam in one frame by a vision sensor, and then detects the seam deviation to adjust the work piece motion adaptively to the seam position sensed by vision sensor. A novel molten pool area image-processing algorithm based on machine vision was proposed. The algorithm processes each image at the speed of 20 frames/second in real-time to extract three feature variables to get the seam deviation. It is proved experimentally that the algorithm is very fast and effective. Issues related to the algorithm are also discussed.

Study on the image processing of laser vision seam tracking system

Seam image processing is the basis of the realization of automatic laser vision seam tracking system, and it has become one of the important research directions. Adding windows processing, gray processing, fast median filtering, binary processing and image edge extraction are used to pretreat the seam image. In the post-processing of seam image, the feature points of the target image are succesfully detected by using center line extraction and feature points detection algorithm based on slope analysis. The whole processing time is less than 150 ms, and the real-time processing of seam image can be implemented.

Development of seam tracking system with ultrasonic sensor using self-tuning fuzzy control

One kind of the SAW seam tracking system with contactless ultrasonic sensor is presented in this paper. The new contactless ultrasonic sensor for seam tracking and the working principle of the seam tracking with the sensor are introduced. Based on the experiments, the optimal values of the fuzzy control parameters α and k 3 are defined by means of the off line adjusting method. Because the self tuning fuzzy control is adopted in the seam tracking system, the overshoot of the system is restrained, the steady state error is reduced, and the system's response speed is improved effectively. The results of the SAW seam tracking experiments show that this system's tracking accuracy is up to ±0.5 mm and the system can satisfy the requirements of the engineering application.

Seam-tracking based on dynamic trajectory planning for a mobile welding robot

A new seam-tracking method based on dynamic trajectory planning for a mobile welding robot is proposed in order to improve the response lag of the mobile robot and the high frequency oscillation in seam-tracking.By using a front-placed laser-based vision sensor to dynamically extract the location of the weld seam in front of torch,the trend and direction of the weld line is roughly obtained.The robot system autonomously and dynamically performs trajectory planning based on the isometric approximation model.Arc sensor technology is applied to detect the offset during welding process in real time.The dynamic compensation of the weld path is done in combination with the control of the mobile robot and the executive body installed on it.Simulated and experimental results demonstrate that the method effectively increases the stability of welding speed and smoothness of the weld track,and hence the weld formation in curves and corners is improved.

Arc sensing system for automatic weld seam tracking (Ⅱ)——Signal processing

Due to violent disturbance of the welding arc, signal processing is the key problem of application of the sensor. By means of the new technique the arc sensor can recognize not only V-groove but also lap joint and butt joint. The sensor has good recognition ability even for welding process with very large current disturbance, e.g. pulsed arc and short circuit welding process, etc. The proposed technique is developed on the basis of modern digital filtering theory and mathematic transformation of the signals from time domain into frequency domain.

Arc sensing system for automatic weld seam tracking(Ⅰ)——Mathematic model

Arc sensor, one of the most popular and effective sensors for automatic seam tracking of arc welding, is systematically studied. A mathematic model of the static and the dynamic properties of the sensor derived on the basis of control theory and experimental method are described. The model extends our understanding of arc sensor from qualitative level to quantitative level. This is important for its design and further development.

The integrated intelligent system for welding seam error and penetration depth identification

A integrated intelligent system for seam tracking and penetration control is given. The system received information of welding seam error and penetration depth from only one sensor, then, it realized seam tracking and penetration control simultaneously. This paper introduces constitution of the system, methods of information recognition, design of the neural fuzzy controller and results practically.

An adaptive thresholding for weld seam image recognition in real time

Image prooessing of wehl seam in real time is an importunity to make welding rohot be able to track weld seam. The algorithm described in this paper combines some image technologies, such as modified Sobel edge detector and Hough transformation function, and especially the thresholds for image processing are ore aled adaptively by Ineans of a neural network. aests proved that this algorithm has a high reliability and rapidity in distinguishing the position of weld seam even with noises. The algorithm can be used ac the basic program .for robot to track welding seam and furthermore for calculating 3 dimensional information plan robot movement automatically.

Welding Deviation Detection Algorithm Based on Extremum of Molten Pool Image Contour

The welding deviation detection is the basis of robotic tracking welding, but the on-line real-time measurement of welding deviation is still not well solved by the existing methods. There is plenty of information in the gas metal arc welding（GMAW） molten pool images that is very important for the control of welding seam tracking. The physical meaning for the curvature extremum of molten pool contour is revealed by researching the molten pool images, that is, the deviation information points of welding wire center and the molten tip center are the maxima and the local maxima of the contour curvature, and the horizontal welding deviation is the position difference of these two extremum points. A new method of weld deviation detection is presented, including the process of preprocessing molten pool images, extracting and segmenting the contours, obtaining the contour extremum points, and calculating the welding deviation, etc. Extracting the contours is the premise, segmenting the contour lines is the foundation, and obtaining the contour extremum points is the key. The contour images can be extracted with the method of discrete dyadic wavelet transform, which is divided into two sub contours including welding wire and molten tip separately. The curvature value of each point of the two sub contour lines is calculated based on the approximate curvature formula of multi-points for plane curve, and the two points of the curvature extremum are the characteristics needed for the welding deviation calculation. The results of the tests and analyses show that the maximum error of the obtained on-line welding deviation is 2 pixels（0.16 ram）, and the algorithm is stable enough to meet the requirements of the pipeline in real-time control at a speed of less than 500 mm/min. The method can be applied to the on-line automatic welding deviation detection.

Dynamic modeling and simulation for nonholonomic welding mobile robot

Based on the Newton-Euler method, the dynamic behaviors of the left and right driving wheels and the robot body for the welding mobile robot were derived. In order to realize the combination control of body turning and slider adjustment, the dynamic behaviors of sliders were also investigated. As a result, a systematic and complete dynamic model for the welding mobile robot was constructed. In order to verify the effectiveness of the above model, a sliding mode tracking control method was proposed and simulated, the lateral error stabilizes between -0.2 mm and ＋0.2 mm, and the total distance of travel for the slider is consistently within 4-2 ram. The simulation results verify the effectiveness of the established dynamic model and also show that the seam tracking controller based on the dynamic model has excellent performance in terms of stability and robustness. Furthermore, the model is found to be very suitable for practical applications of the welding mobile robot.

Study on intelligent welding mobile robot with the function of auto-searching weld line

The development of welding robots suitable for specially unstructured working enviroments has been become an important development direction of industrial robot application because large-scale welding structures have been used more and more widely in modern industry. In this paper, an intelligent mobile robot for welding of ship deck with the function of autosearching weld line was presented. A wheeled motion mechanism and a cross adjustment slider are used for the welding robot body. A sensing system based on laser-PSD （position sensitive detector） displacement sensor was developed to obtain two dimensional deviation signals during seam tracking. A full-digital control system based on DSP and CPLD has also been realized to implement complex and high-performance control algorithms. Furthermore, the system has still the function of auto-searching weld line according to the characteristics information of weld groove and adjusting posture itself to the desired status preparing for welding. The experiment of auto-searching welding line shows that the robot has high tracing accuracy, and can satisfy the requirement of practical welding project.

AR-assisted robot welding programming

Robotic welding demands high accuracy and precision.However,robot programming is often a tedious and time-consuming process that requires expert knowledge.This paper presents an augmented reality assisted robot welding task programming(ARWP)system using a user-friendly augmented reality(AR)interface that simplifies and speeds up the programming of robotic welding tasks.The ARWP system makes the programming of robot welding tasks more user-friendly and reduces the need for trained programmers and expertise in specific robotic systems.The AR interface simplifies the definition of a welding path as well as the welding gun orientation,and the system;the system can locate the welding seam of a workpiece quickly and generate a viable welding path based on the user input.The developed system is integrated with the touch-sensing capability of welding robots in order to locate the welding path accurately based on the user input,for fillet welding.The system is applicable to other welding processes and methods of seam localization.The system implementation is described and evaluated with a case study.