Novel Real-Time Seam Tracking Algorithm Based on Vector Angle and Least Square Method

Real-time seam tracking can improve welding quality and enhance welding efficiency during the welding process in automobile manufacturing.However,the teaching-playing welding process,an off-line seam tracking method,is still dominant in automobile industry,which is less flexible when welding objects or situation change.A novel real-time algorithm consisting of seam detection and generation is proposed to track seam.Using captured 3D points,space vectors were created between two adjacent points along each laser line and then a vector angle based algorithm was developed to detect target points on the seam.Least square method was used to fit target points to a welding trajectory for seam tracking.Furthermore,the real-time seam tracking process was simulated in MATLAB/Simulink.The trend of joint angles vs.time was logged and a comparison between the off-line and the proposed seam tracking algorithm was conducted.Results show that the proposed real-time seam tracking algorithm can work in a real-time scenario and have high accuracy in welding point positioning.

A machine vision approach to seam tracking in real-time in PAW of large-diameter stainless steel tube

Manual monitoring and seam tracking through watching weld pool images in real-time, by naked eyes or by industrial TV, are experience-depended, subjective, labor intensive, and sometimes biased. So it is necessary to realize the automation of computer-aided seam tracking. A PAW (plasma arc welding) seam tracking system was developed, which senses the molten pool and the seam in one frame by a vision sensor, and then detects the seam deviation to adjust the work piece motion adaptively to the seam position sensed by vision sensor. A novel molten pool area image-processing algorithm based on machine vision was proposed. The algorithm processes each image at the speed of 20 frames/second in real-time to extract three feature variables to get the seam deviation. It is proved experimentally that the algorithm is very fast and effective. Issues related to the algorithm are also discussed.

Real-time accurate hand path tracking and joint trajectory planning for industrial robots(Ⅰ)

Previously, researchers raised the accuracy for a robot′s hand to track a specified path in Car-tesian space mainly through increasing the number of knots on the path and the number of the path′s segments, which results in the heavier online computational burden for the robot controller. Aiming at overcoming this drawback, the authors propose a new kind of real-time accurate hand path tracking and joint trajectory planning method. Through selecting some extra knots on the specified hand path by a certain rule and introducing a sinusoidal function to the joint displacement equation of each segment, this method can greatly raise the path tracking accuracy of robot′s hand and does not change the number of the path′s segments. It also does not increase markedly the computational burden of robot controller. The result of simulation indicates that this method is very effective, and has important value in increasing the application of industrial robots.

A Real-time Face/Hand Tracking Method for Chinese Sign Language Recognition

This paper introduces a new Chinese Sign Language recognition (CSLR) system and a method of real time tracking face and hand applied in the system. In the method, an improved agent algorithm is used to extract the region of face and hand and track them. Kalman filter is introduced to forecast the position and rectangle of search, and self adapting of target color is designed to counteract the effect of illumination.

Real-time accurate hand path tracking and joint trajectory planning for industrial robots(Ⅱ)

Previously, researchers raised the accuracy for a robot′s hand to track a specified path in Cartesian space mainly through increasing the number of knots on the path and the segments of the path. But, this method resulted in the heavier on line computational burden for the robot controller. In this paper, aiming at this drawback, the authors propose a new kind of real time accurate hand path tracking and joint trajectory planning method for robots. Through selecting some extra knots on the specified hand path by a certain rule, which enables the number of knots on each segment to increase from two to four, and through introducing a sinusoidal function and a cosinoidal function to the joint displacement equation of each segment, this method can raise the path tracking accuracy of robot′s hand greatly but does not increase the computational burden of robot controller markedly.

Real-time tracking of deformable objects based on MOK algorithm

The traditional oriented FAST and rotated BRIEF（ORB） algorithm has problems of instability and repetition of keypoints and it does not possess scale invariance. In order to deal with these drawbacks, a modified ORB（MORB） algorithm is proposed. In order to improve the precision of matching and tracking, this paper puts forward an MOK algorithm that fuses MORB and Kanade-Lucas-Tomasi（KLT）. By using Kalman, the object＇s state in the next frame is predicted in order to reduce the size of search window and improve the real-time performance of object tracking. The experimental results show that the MOK algorithm can accurately track objects with deformation or with background clutters, exhibiting higher robustness and accuracy on diverse datasets. Also, the MOK algorithm has a good real-time performance with the average frame rate reaching 90.8 fps.

Real-time localization for underwater equipment using an extremely low frequency electric field

A new real-time underwater equipment location method adopting an electric field induced by a standard current source is proposed.Our goals were real-time tracking and location of stationary or moving underwater equipment both in shallow and deep seas,under noisy conditions.The main features of this method are as follows:(1)a standard current source on the water surface,which can be towed by a vehicle,consisting of two electrodes,a signal generator,and a GPS unit;(2)measurement of the extremely low frequency(ELF)electric field emitted by the current source,made possible by electric field sensors on the underwater equipment;(3)position of the underwater equipment is estimated in real time based on a progressive update extended Kalman filter(PUEKF),which is carried out using the propagation model of an ELF electric field because the electric field at the position of the underwater equipment and the current source position are known.We verified the accuracy of our method and confirmed real-time location feasibility through numerical,physical scale,and real-time sea experiments.Through numerical experiments,we verified that our method works for underwater equipment location in real-world conditions,and the location error can be less than 0.2 m.Next,real-time location experiments for stationary underwater measuring equipment in water tank were conducted.The result shows that the location error can be less than 0.1 m.We also confirmed real-time location feasibility through the use of offshore experiment.We expect that our method will complement conventional underwater acoustic location methods for underwater equipment in acoustically noisy environments.

Design of Robot Welding Seam Tracking System with Structured Light Vision

Robot welding is an important developing direction of welding automation and intelligentization, and automatic seam tracking technology is one of principal research domains. Nowadays, seam tracking system with structured light vision becomes a hot research. Structured light vision seam tracking products abroad are generally very expensive and can only be applied on special occasions. In China, the research of structured light vision seam tracking system is still just on the stage of experiments. A robot real-time seam tracking system with line structured light vision is designed. The hardware system is set up, a filtering method for line structure seam image is improved, and compared with common filtering, it has better effect and characteristic of real time. Two methods, fast template matching and fast Hough transform, to recognize the image coordinates of seam center are improved. Two new image recognition methods, structure element matching and comer detecting, are proposed. The comparison of seam image recognition shows that fast template matching and comer detecting are more precise and stable than the other two methods, and comer detecting is the best in real time. A simultaneous calibration for camera parameters and robot hand-eye is also proposed, and calculation shows that the calibration is effective and feasible. The robot seam tracking tests for linear and folded lap-joint are performed, which are based on the above four image recognition methods, and the results indicate that four image recognition methods are all applicable to real-time seam tracking, and the whole system sufficed for the requirements of real-time seam tracking. Automatic seam tracking with line structured light vision is feasible and has good versatility.

Seam Tracking Technology for Hyperbaric Underwater Welding

Automatic weld seam tracking technology to be used in hyperbaric underwater damaged pipeline repair welding is much more important, because of poor bevel preparation and severe working condition. A weld seam tracking system based on digital signal processing（DSP） passive light weld image processing technology has been established. A convenient charge coupled device（CCD） camera system was used in the high pressure environment with the help of an aperture and focus altering mechanism to guarantee overall image visibility in the scope of pressure below 0.7 MPa. The system can be used in the hyperbaric environment to pick up the real welding image of both the welding arc and the welding pool. The newly developed DSP technology was adopted to achieve the goal of system real time characteristics. An effective weld groove edge recognition technique including narrow interesting window opening, middle value wave filtering, Sobel operator weld edge detecting and edge searching in a defined narrow area was proposed to remove the guide error and system accuracy was ensured. The results of tracking simulation and real tracking application with arc striking have proved the validity and the accuracy of the mentioned system and the image processing method.

Seam Tracking and Visual Control for Robotic Arc Welding Based on Structured Light Stereovision

A real-time arc welding robot visual control system based on a local network with a multi-level hierarchy is developed in this paper. It consists of an intelligence and human-machine interface level, a motion planning level, a motion control level and a servo control level. The last three levels form a local real-time open robot controller, which realizes motion planning and motion control of a robot. A camera calibration method based on the relative movement of the end-effector connected to a robot is proposed and a method for tracking weld seam based on the structured light stereovision is provided. Combining the parameters of the cameras and laser plane, three groups of position values in Cartesian space are obtained for each feature point in a stripe projected on the weld seam. The accurate three-dimensional position of the edge points in the weld seam can be calculated from the obtained parameters with an information fusion algorithm. By calculating the weld seam parameter from position and image data, the movement parameters of the robot used for tracking can be determined. A swing welding experiment of type V groove weld is successfully conducted, the results of which show that the system has high resolution seam tracking in real-time, and works stably and efficiently.

Seam-tracking based on dynamic trajectory planning for a mobile welding robot

A new seam-tracking method based on dynamic trajectory planning for a mobile welding robot is proposed in order to improve the response lag of the mobile robot and the high frequency oscillation in seam-tracking.By using a front-placed laser-based vision sensor to dynamically extract the location of the weld seam in front of torch,the trend and direction of the weld line is roughly obtained.The robot system autonomously and dynamically performs trajectory planning based on the isometric approximation model.Arc sensor technology is applied to detect the offset during welding process in real time.The dynamic compensation of the weld path is done in combination with the control of the mobile robot and the executive body installed on it.Simulated and experimental results demonstrate that the method effectively increases the stability of welding speed and smoothness of the weld track,and hence the weld formation in curves and corners is improved.

Study of image processing for V-shape groove and robotic weld seam tracking based on laser vision

Single-stripe laser was applied to acquire V-shape groove contour information. Most of arc light and splash noise was removed and stripe laser image was kept by wavelet transform. Half-threshold algorithm was used for image segmentation and stripe laser image was gotten after refining. Weld seam center position was identified and extracted by extreme curvature corner detection method. The location of torch was detected to accord the frequency of computer program with robot program and serial communication program. The tracking experiments of sidelong, reflex and curve weld line show that the system can meet the demand of the tracking precision under normal welding conditions.

The characteristics of welding currents in a rotating arc leadtandem GMAW process and the weld-seam tracking

Tandem gas metal arc welding （ T-GMAW） process shows a high deposition rate that up to three times o f the single electrode GMAW, so the welding speed could be significantly increased in this process. However, the majority o f this process applications are based on the pre-programmed robotic welding, which does not allow them to track the seam real-time during welding. Rotating arc sensor, sensing the seam position by detecting the changing of welding currents, has been widely adopted in the automatic robot welding process. It is proposed in this paper to integrate the rotating arc sensor with a trailing torch to develop a new approach of rotating arc lead tandem gas metal arc welding （RLT-GMAW） process. The characteristics of the welding currents in the proposed new welding process were firstly studied, and then a self-turning fuzzy control seam tracking strategy was developed for the mobile robot automatic welding. The experimental results showed that the proposed RLT-GMAW process had an excellent seam tracking performance and high welding deposition rate. Even if there were some electromagnetic interactions between the two arcs, the deviation of the welding seam could also be reflected by the fluctuation of the welding currents on the leading arc once the correct welding parameters were selected. Based on the detected deviation, the welding tracking experiments showed that the proposed self-turning fuzzy controller had a good performance for the RLT-GMAW process seam tracking.

Study on contactless ultrasonic sensor and seam tracking

The sensitivity of contactless ultrasonic sensor is improved from 3.3% to 26% through acoustic and electric matching technique in this paper.Along dual crest lines auto seam tracking is firstly realized with contactless ultrasonic sensing.The focused acoustic lens is accurately designed, and a sonic beam with 0.5 mm in diameter is achieved. By means of software and hardware technique the accuracy of seam tracking with ultrasonic sensing is 0.5 mm in lateral direction and 0.2 mm in longitudinal direction respectively.

Multiple-Object Tracking Using Histogram Stamp Extraction in CCTV Environments

Object tracking,an important technology in the field of image processing and computer vision,is used to continuously track a specific object or person in an image.This technology may be effective in identifying the same person within one image,but it has limitations in handling multiple images owing to the difficulty in identifying whether the object appearing in other images is the same.When tracking the same object using two or more images,there must be a way to determine that objects existing in different images are the same object.Therefore,this paper attempts to determine the same object present in different images using color information among the unique information of the object.Thus,this study proposes a multiple-object-tracking method using histogram stamp extraction in closed-circuit television applications.The proposed method determines the presence or absence of a target object in an image by comparing the similarity between the image containing the target object and other images.To this end,a unique color value of the target object is extracted based on its color distribution in the image using three methods:mean,mode,and interquartile range.The Top-N accuracy method is used to analyze the accuracy of each method,and the results show that the mean method had an accuracy of 93.5%(Top-2).Furthermore,the positive prediction value experimental results show that the accuracy of the mean method was 65.7%.As a result of the analysis,it is possible to detect and track the same object present in different images using the unique color of the object.Through the results,it is possible to track the same object that can minimize manpower without using personal information when detecting objects in different images.In the last response speed experiment,it was shown that when the mean was used,the color extraction of the object was possible in real time with 0.016954 s.Through this,it is possible to detect and track the same object in real time when using the proposed method.

Study on the image processing of laser vision seam tracking system

Seam image processing is the basis of the realization of automatic laser vision seam tracking system, and it has become one of the important research directions. Adding windows processing, gray processing, fast median filtering, binary processing and image edge extraction are used to pretreat the seam image. In the post-processing of seam image, the feature points of the target image are succesfully detected by using center line extraction and feature points detection algorithm based on slope analysis. The whole processing time is less than 150 ms, and the real-time processing of seam image can be implemented.

Development of seam tracking system with ultrasonic sensor using self-tuning fuzzy control

One kind of the SAW seam tracking system with contactless ultrasonic sensor is presented in this paper. The new contactless ultrasonic sensor for seam tracking and the working principle of the seam tracking with the sensor are introduced. Based on the experiments, the optimal values of the fuzzy control parameters α and k 3 are defined by means of the off line adjusting method. Because the self tuning fuzzy control is adopted in the seam tracking system, the overshoot of the system is restrained, the steady state error is reduced, and the system's response speed is improved effectively. The results of the SAW seam tracking experiments show that this system's tracking accuracy is up to ±0.5 mm and the system can satisfy the requirements of the engineering application.

Seam tracking control for mobile welding robot based on vision sensor

To solve the seam tracking problem of mobile welding robot,a new controller based on the dynamics of mobile welding robot was designed using the method of backstepping kinematics into dynamics.A self-turning fuzzy controller and a fuzzy-Gaussian neural network(FGNN) controller were designed to complete coordinately controlling of cross-slider and wheels.The fuzzy-neural control algorithm was described by applying the Gaussian function and back propagation(BP) learning rule was used to tune the membership function in real time by applying the FGNN controller.To make the tracking more quickly and smoothly,the neural network controller based on dynamic model was designed,which utilized self-learning and self-adaptive ability of the neural network to deal with the partial uncertainty and the disturbances of the parameters of the robot dynamic model and real-time compensate the dynamics coupling.The results show that the selected control input torques make the system globally and asymptotically stable based on the Lyapunov function selected out;the accuracy of the proposed controller tracing is within ±0.4 mm and can satisfy the requirements of practical welding project.

High-speed railway track components inspection framework based on YOLOv8 with high-performance model deployment

Railway inspection poses significant challenges due to the extensive use of various components in vast railway networks,especially in the case of high-speed railways.These networks demand high maintenance but offer only limited inspection windows.In response,this study focuses on developing a high-performance rail inspection system tailored for high-speed railways and railroads with constrained inspection timeframes.This system leverages the latest artificial intelligence advancements,incorporating YOLOv8 for detection.Our research introduces an efficient model inference pipeline based on a producer-consumer model,effectively utilizing parallel processing and concurrent computing to enhance performance.The deployment of this pipeline,implemented using C++,TensorRT,float16 quantization,and oneTBB,represents a significant departure from traditional sequential processing methods.The results are remarkable,showcasing a substantial increase in processing speed:from 38.93 Frames Per Second(FPS)to 281.06 FPS on a desktop system equipped with an Nvidia RTX A6000 GPU and from 19.50 FPS to 200.26 FPS on the Nvidia Jetson AGX Orin edge computing platform.This proposed framework has the potential to meet the real-time inspection requirements of high-speed railways.

Visual Tracking System for Welding Seams

为了追踪在容器焊接缝接的狭窄的靶子，生产，一个视觉追踪系统在这份报纸基于聪明的照相机被建议。一个聪明的照相机被用作传感器检测焊接缝接。特征抽取算法与聪明的照相机的特征的考虑被设计，它被用来计算在焊接火炬和焊接缝接之间的错误。视觉控制系统基于图象被介绍，它采用一个可编程的控制器控制一台 stepper 马达消除聪明的照相机检测的追踪的错误。实验被进行表明视觉系统的有效性。