Calibration method for multi-line structured light vision sensor based on Plücker line

For the rapid calibration of multi-line structured light system,a method based on Plücker line was proposed.Most of the conventional line-structured light calibration methods extract the feature points and transform the coordinates of points to obtain the plane equation.However,a large number of points lead to complicated operation which is not suitable for the application scenarios of multi-line structured light.To solve this issue,a new calibration method was proposed that applied the form of Plücker matrix throughout the whole calibration process,instead of using the point characteristics directly.The advantage of this method is that the light plane equation can be obtained quickly and accurately in the camera coordinate frame.Correspondingly a planar target particularly for calibrating multi-line structured light was also designed.The regular lines were transformed into Plücker lines by extending the two-dimensional image plane and defining a new image space.To transform the coordinate frame of Plücker lines,the perspective projection mathematical model was re-expressed based on the Plücker matrix.According to the properties of the line and plane in the Plücker space,a linear matrix equation was efficiently constructed by combining the Plücker matrices of several coplanar lines so that the line-structured light plane equation could be furtherly solved.The experiments performed validate the proposed method and demonstrate the significant improvement in the calibration accuracy,when the test distance is 1.8 m,the root mean square(RMS)error of the three-dimensional point is within 0.08 mm.

Generation of structured light beams with polarization variation along arbitrary spatial trajectories using tri-layer metasurfaces

Conventionally,the spatially structured light beams produced by metasurfaces primarily highlight the polarization modulation of the beams propagating along the optical axis or the beams'spatial transmission trajectory.In particular,along the optical axis,the polarization state is either constant or varies continuously in each output plane.Here,we develop innovative spatially structured light beams with continually changing polarization along any arbitrary spatial transmission trajectories.With tri-layer metallic metasurfaces,the geometric characteristics of each layer structure can be adjusted to modulate the phase and polarization state of the incident terahertz(THz)wave.The beam will converge to the predefined trajectory along several paths to generate a Bessel-like beam with longitudinal polarization changes.We demonstrate the versatility of the approach by designing two THz-band structured light beams with varying polarization states along the spatial helical transmission trajectory.Continuous linear polarization changes and linear polarization to right circular polarization(RCP)and back to linear polarization changes are realized respectively.The experimental results are basically consistent with the simulated results.Our proposal for arbitrary trajectory structured light beams with longitudinally varying polarization offers a practical method for continuously regulating the characteristics of spatial structured light beams with non-axial transmission.This technique has potential uses in optical encryption,particle manipulation,and biomedical imaging.

SEGMENTATION ALGORITHM BASED ON EDGE-SEARCHING FOR MULTI-LINEAR STRUCTURED LIGHT IMAGES

Aiming at the problem that the existence of disturbances on the edges of light-stripe makes the segmentation of the light-stripes images difficult, a new segmentation algorithm based on edge-searching is presented. It firstly calculates every edge pixel＇s horizontal coordinate grads to produce the corresponding grads-edge, then uses a designed length-variable l D template to scan the light-stripes＇ grads-edges. The template is able to find the disturbances with different width utilizing the distributing character of the edge disturbances. The found disturbances are eliminated finally. The algorithm not only can smoothly segment the light-stripes images, but also eliminate most disturbances on the light-stripes＇ edges without damaging the light-stripes images＇ 3D information. A practical example of using the proposed algorithm is given in the end. It is proved that the efficiency of the algorithm has been improved obviously by comparison.

Nonlinear effect of the structured light profilometry in the phase-shifting method and error correction

Digital structured light （SL） profilometry is increasingly used in three-dimensional （3D） measurement technology. However, the nonlinearity of the off-the-shelf projectors and cameras seriously reduces the measurement accuracy. In this paper, first, we review the nonlinear effects of the projector-camera system in the phase-shifting structured light depth measurement method. We show that high order harmonic wave components lead to phase error in the phase-shifting method. Then a practical method based on frequency domain filtering is proposed for nonlinear error reduction. By using this method, the nonlinear calibration of the SL system is not required. Moreover, both the nonlinear effects of the projector and the camera can be effectively reduced. The simulations and experiments have verified our nonlinear correction method.

Seam Tracking and Visual Control for Robotic Arc Welding Based on Structured Light Stereovision

A real-time arc welding robot visual control system based on a local network with a multi-level hierarchy is developed in this paper. It consists of an intelligence and human-machine interface level, a motion planning level, a motion control level and a servo control level. The last three levels form a local real-time open robot controller, which realizes motion planning and motion control of a robot. A camera calibration method based on the relative movement of the end-effector connected to a robot is proposed and a method for tracking weld seam based on the structured light stereovision is provided. Combining the parameters of the cameras and laser plane, three groups of position values in Cartesian space are obtained for each feature point in a stripe projected on the weld seam. The accurate three-dimensional position of the edge points in the weld seam can be calculated from the obtained parameters with an information fusion algorithm. By calculating the weld seam parameter from position and image data, the movement parameters of the robot used for tracking can be determined. A swing welding experiment of type V groove weld is successfully conducted, the results of which show that the system has high resolution seam tracking in real-time, and works stably and efficiently.

Nonlinear optics with structured light

The interest in tailoring light in all its degrees of freedom is steadily gaining traction,driven by the tremendous developments in the toolkit for the creation,control and detection of what is now called structured light.Because the complexity of these optical fields is generally understood in terms of interference,the tools have historically been linear optical elements that create the desired superpositions.For this reason,despite the long and impressive history of nonlinear optics,only recently has the spatial structure of light in nonlinear processes come to the fore.In this review we provide a concise theoretical framework for understanding nonlinear optics in the context of structured light,offering an overview and perspective on the progress made,and the challenges that remain.

On-line detection system of weld-based rapid prototyping process based on structured light

In order to realize the closed-loop control for rapid prototyping process based on gas metal arc welding, the geometric parameters of weld beads should be detected. In this study, a vision sensor system consisting of a linear laser projector and a CCD camera was designed to collect images of weld beads. Then, an image processing approach which combines with a Gaassian filter and an improved gravity method was used to extract the centerline of a light stripe based on VC ＋＋. Feature points of the centerline were identical directly by means of an image fusion algorithm. Experimental results show that image fusion is an effective approach to measure the width and height of the weld bead with high accuracy. This method can identify beads effectively in multi-pass welding and avoid designing different modes to suit all kinds of shapes.

Generation of structure light in probe absorption spectrum via microwave-driven Y-type atomic system

Behavior of structure light is investigated by monitoring probe absorption using a microwave-driven Y-type atomic media configuration.The system under consideration is driven by one of the control vortex beams as well as an extra non-vortex control beam to ensure electromagnetically induced transparency.The significant aspect in the generation of structured light is the azimuthal phase-dependent modification for probe absorption.Further intensity distribution for absorption spectra is examined for simultaneously evaluating both the control vortex beams.We also go through the radial distribution of intensity for various orbital angular momentum values.Different modes of structural beams may be distinguished using the suggested approach.Our research gives us a way for rapidly transferring vortex wavefronts from control field to probe absorption profile.This could be useful in quantum information processing.

Structured Light 3D Vision Based on Laser Scanning Space-encoding and Its Synchro Control

This paper presents the principle and mathematic model for the 3D depth map method based on space encoding images performed by modulating scanning structuredlight according to time sequences,and the synchro control among the camera,laser diode modulation and scanning polyhedron.

3D surface reconstruction based on binocular vision using structured light

A 3D surface reconstruction method using a binocular stereo vision technology and a coded structured light,which combines a gray code with phase-shift has been studied.The accuracy of the 3 D surface reconstruction mainly depends on the decoding of gray code views and phase-shift views.In order to find the boundary accurately,gray code patterns and their inverses are projected onto a human eye plaster model.The period dislocation between the gray code views and the phase-shift views in the course of decoding has been analyzed and a new method has been proposed to solve it.The splicing method is based on feature points.The result of the 3D surface reconstruction shows the accuracy and reliability of our method.

Construction for M-arrays and application in structured light

M-arrays are random arrays in which an appropriate sub-window appears only once in the whole array. Coded structured light based on M-arrays is one=shot technique to rapidly acquire 3D information of unknown surfaces by projecting suitable patterns onto a measuring surface. This paper presents a method to construct large size M-arrays based on the piece growing algorithm in which an array is constructed by many pieces through splicing each other. Reconstructing 3D shapes by utilizing the designed pattern based on constructed M-arrays for two objects are given.

Three-Dimensional Measurement Using Structured Light Based on Deep Learning

Three-dimensional(3D)reconstruction using structured light projection has the characteristics of non-contact,high precision,easy operation,and strong real-time performance.However,for actual measurement,projection modulated images are disturbed by electronic noise or other interference,which reduces the precision of the measurement system.To solve this problem,a 3D measurement algorithm of structured light based on deep learning is proposed.The end-to-end multi-convolution neural network model is designed to separately extract the coarse-and fine-layer features of a 3D image.The point-cloud model is obtained by nonlinear regression.The weighting coefficient loss function is introduced to the multi-convolution neural network,and the point-cloud data are continuously optimized to obtain the 3D reconstruction model.To verify the effectiveness of the method,image datasets of different 3D gypsum models were collected,trained,and tested using the above method.Experimental results show that the algorithm effectively eliminates external light environmental interference,avoids the influence of object shape,and achieves higher stability and precision.The proposed method is proved to be effective for regular objects.

Research on Visual Detection Algorithm for Groove Feature Sizes by Means of Structured Light Projection

The visual inspection is an economical and effective method for welding. For measuring the feature sizes of grooves,a method based on line structured light is presented. Firstly,an adaptive algorithm to extract the subpixel centerline of structured light stripes is introduced to deal with the uneven width and grayscale distributions of laser stripes,which is based on the quadratic weighted grayscale centroid. By means of region-of-interest(ROI)division and image difference,an image preprocessing algorithm is developed for filtering noise and improving image quality. Furthermore,to acquire geometrical dimensions of various grooves and groove types precisely,the subpixel feature point extraction algorithm of grooves is designed. Finally, experimental results of feature size measuring show that the absolute error of measurement is 0.031—0.176 mm,and the relative error of measurement is 0.2%—3.6%.

Calibration of line structured light vision system based on camera’s projective center

Based on the characteristics of line structured light sensor, a speedy method for the calibration was established. With the coplanar reference target, the spacial pose between camera and optical plane can be calibrated by using of the camera’s projective center and the light’s information in the camera’s image surface. Without striction to the movement of the coplanar reference target and assistant adjustment equipment, this calibration method can be implemented. This method has been used and decreased the cost of calibration equipment, simplified the calibration procedure, improved calibration efficiency. Using experiment, the sensor can attain relative accuracy about 0.5%, which indicates the rationality and effectivity of this method.

MULTIPARAMETER MEASUREMENT FOR RACEWAY GROOVE OF BEARING BASED ON 3D RECONSTRUCTION WITH DIGITAL STRUCTURED LIGHT

A fast 3D reconstruction method based on structured light to measure various parameters of the raceway groove is presented. Digital parallel grating stripes distributed with sine density are projected onto the raceway groove by a DLP projector, and distorting of stripes is happened on the raceway. Simultaneously, aided by three-step phase-shifting approach, three images covered by different stripes are obtained by a high-resolution CCD camera at the same location, thus a more accuracy local topography can be obtained. And then the bearing is rotated on a high precision computer controlled rotational stage. Three images are also obtained as the former step at next planned location triggered by the motor. After one cycle, all images information is combined through the mosaics. As a result, the 3D information of raceway groove can be gained. Not only geometric properties but also surface flaws can be extracted by software. A preliminary hardware system has been built, with which some geometric parameters have been extracted from reconstructed local topography.

Application of color structured light pattern to measurement of large out-of-plane deformation

Measurement of out-of-plane deformation is significant to understanding of the deflection mechanisms of the plate and tube structures.In this study,a new surface contouring technique with color structured light is applied to measure the out-of-plane deformation of structures with one-shot projection.Through color fringe recognizing,decoding and triangulation processing for the captured images corresponding to each deformation state,the feasibility of the method is testified by the measurement of elastic deflections of a flexible square plate,showing good agreement with those from the calibrated displacement driver.The plastic deformation of two alloy aluminum rectangular tubes is measured to show the technique application to surface topographic evaluation of the buckling structures with large displacements.

Hybrid Visual Servoing Control for Robotic Arc Welding Based on Structured Light Vision

A novel hybrid visual servoing control method based on structured light vision is pro-posed for robotic arc welding with a general six degrees of freedom robot. It consists of a positioncontrol inner-loop in Cartesian space and two outer-loops. One is position-based visual control inCartesian space for moving in the direction of weld seam, i.e., weld seam tracking, another is image-based visual control in image space for adjustment to eliminate the errors in the process of tracking.A new Jacobian matrix from image space of the feature point on structured light stripe to Cartesianspace is provided for dierential movement of the end-e?ector. The control system model is simplifiedand its stability is discussed. An experiment of arc welding protected by gas CO2 for verifying iswell conducted.

Calibration for 3D Structured Light Measurement

A calibration procedure was developed for three-dimensional(3D) binocular structured light measurement systems. In virtue of a specially designed pattern, matching points in stereo images are extracted. And then sufficient 3D space points are obtained through pairs of images with the intrinsic and extrinsic parameters of each camera estimated prior and consequently some lights are calibrated by means of multi point fitting. Finally, a mathematical model is applied to interpolate and approximate all dynamic scanning lights based on geometry. The process of calibration method is successfully used in the binocular 3D measurement system based on structured lights and the 3D reconstruction results are satisfying.

Influence of sampling on face measuring system based on composite structured light

Human face can be rebuilt to a three-dimensional (3 D) digital profile based on an optical 3D sensing system named Composite Fourier-Transform Profilometry (CFTP) where a composite structured light will be used. To study the sampling effect during the digitization process in practical CFTP, the pectinate function and convolution theorem were introduced to discuss the potential phase errors caused by sampling the composite pattern along two orthogonal directions. The selecting criterions of sampling frequencies are derived and the results indicate that to avoid spectral aliasing, the sampling frequency along the phrase variation direction must be at least four times as the baseband and along the orthogonal direction it must be at least three times as the larger frequency of the two carrier frequencies. The practical experiment of a model face reconstruction verified the theories.