Nonlinear effect of the structured light profilometry in the phase-shifting method and error correction

Digital structured light （SL） profilometry is increasingly used in three-dimensional （3D） measurement technology. However, the nonlinearity of the off-the-shelf projectors and cameras seriously reduces the measurement accuracy. In this paper, first, we review the nonlinear effects of the projector-camera system in the phase-shifting structured light depth measurement method. We show that high order harmonic wave components lead to phase error in the phase-shifting method. Then a practical method based on frequency domain filtering is proposed for nonlinear error reduction. By using this method, the nonlinear calibration of the SL system is not required. Moreover, both the nonlinear effects of the projector and the camera can be effectively reduced. The simulations and experiments have verified our nonlinear correction method.

Seam Tracking and Visual Control for Robotic Arc Welding Based on Structured Light Stereovision

A real-time arc welding robot visual control system based on a local network with a multi-level hierarchy is developed in this paper. It consists of an intelligence and human-machine interface level, a motion planning level, a motion control level and a servo control level. The last three levels form a local real-time open robot controller, which realizes motion planning and motion control of a robot. A camera calibration method based on the relative movement of the end-effector connected to a robot is proposed and a method for tracking weld seam based on the structured light stereovision is provided. Combining the parameters of the cameras and laser plane, three groups of position values in Cartesian space are obtained for each feature point in a stripe projected on the weld seam. The accurate three-dimensional position of the edge points in the weld seam can be calculated from the obtained parameters with an information fusion algorithm. By calculating the weld seam parameter from position and image data, the movement parameters of the robot used for tracking can be determined. A swing welding experiment of type V groove weld is successfully conducted, the results of which show that the system has high resolution seam tracking in real-time, and works stably and efficiently.

On-line detection system of weld-based rapid prototyping process based on structured light

In order to realize the closed-loop control for rapid prototyping process based on gas metal arc welding, the geometric parameters of weld beads should be detected. In this study, a vision sensor system consisting of a linear laser projector and a CCD camera was designed to collect images of weld beads. Then, an image processing approach which combines with a Gaassian filter and an improved gravity method was used to extract the centerline of a light stripe based on VC ＋＋. Feature points of the centerline were identical directly by means of an image fusion algorithm. Experimental results show that image fusion is an effective approach to measure the width and height of the weld bead with high accuracy. This method can identify beads effectively in multi-pass welding and avoid designing different modes to suit all kinds of shapes.

Generation of structure light in probe absorption spectrum via microwave-driven Y-type atomic system

Behavior of structure light is investigated by monitoring probe absorption using a microwave-driven Y-type atomic media configuration.The system under consideration is driven by one of the control vortex beams as well as an extra non-vortex control beam to ensure electromagnetically induced transparency.The significant aspect in the generation of structured light is the azimuthal phase-dependent modification for probe absorption.Further intensity distribution for absorption spectra is examined for simultaneously evaluating both the control vortex beams.We also go through the radial distribution of intensity for various orbital angular momentum values.Different modes of structural beams may be distinguished using the suggested approach.Our research gives us a way for rapidly transferring vortex wavefronts from control field to probe absorption profile.This could be useful in quantum information processing.

Nonlinear optics with structured light

The interest in tailoring light in all its degrees of freedom is steadily gaining traction,driven by the tremendous developments in the toolkit for the creation,control and detection of what is now called structured light.Because the complexity of these optical fields is generally understood in terms of interference,the tools have historically been linear optical elements that create the desired superpositions.For this reason,despite the long and impressive history of nonlinear optics,only recently has the spatial structure of light in nonlinear processes come to the fore.In this review we provide a concise theoretical framework for understanding nonlinear optics in the context of structured light,offering an overview and perspective on the progress made,and the challenges that remain.

Structured Light 3D Vision Based on Laser Scanning Space-encoding and Its Synchro Control

ＳｔｒｕｃｔｕｒｅｄＬｉｇｈｔ３ＤＶｉｓｉｏｎＢａｓｅｄｏｎＬａｓｅｒＳｃａｎｎｉｎｇＳｐａｃｅ┐ｅｎｃｏｄｉｎｇａｎｄＩｔｓＳｙｎｃｈｒｏＣｏｎｔｒｏｌ＊ＹＵＸｉａｏｙａｎｇ（于晓洋）ＱＩＡＮＧＸｉｆｕ（强锡富）ＺＨＡＮＧＪｉａｎ＊＊（张健）ＷＵＬ...

SEGMENTATION ALGORITHM BASED ON EDGE-SEARCHING FOR MULTI-LINEAR STRUCTURED LIGHT IMAGES

Aiming at the problem that the existence of disturbances on the edges of light-stripe makes the segmentation of the light-stripes images difficult, a new segmentation algorithm based on edge-searching is presented. It firstly calculates every edge pixel＇s horizontal coordinate grads to produce the corresponding grads-edge, then uses a designed length-variable l D template to scan the light-stripes＇ grads-edges. The template is able to find the disturbances with different width utilizing the distributing character of the edge disturbances. The found disturbances are eliminated finally. The algorithm not only can smoothly segment the light-stripes images, but also eliminate most disturbances on the light-stripes＇ edges without damaging the light-stripes images＇ 3D information. A practical example of using the proposed algorithm is given in the end. It is proved that the efficiency of the algorithm has been improved obviously by comparison.

3D surface reconstruction based on binocular vision using structured light

A 3D surface reconstruction method using a binocular stereo vision technology and a coded structured light,which combines a gray code with phase-shift has been studied.The accuracy of the 3 D surface reconstruction mainly depends on the decoding of gray code views and phase-shift views.In order to find the boundary accurately,gray code patterns and their inverses are projected onto a human eye plaster model.The period dislocation between the gray code views and the phase-shift views in the course of decoding has been analyzed and a new method has been proposed to solve it.The splicing method is based on feature points.The result of the 3D surface reconstruction shows the accuracy and reliability of our method.

Construction for M-arrays and application in structured light

M-arrays are random arrays in which an appropriate sub-window appears only once in the whole array. Coded structured light based on M-arrays is one=shot technique to rapidly acquire 3D information of unknown surfaces by projecting suitable patterns onto a measuring surface. This paper presents a method to construct large size M-arrays based on the piece growing algorithm in which an array is constructed by many pieces through splicing each other. Reconstructing 3D shapes by utilizing the designed pattern based on constructed M-arrays for two objects are given.

Three-Dimensional Measurement Using Structured Light Based on Deep Learning

Three-dimensional(3D)reconstruction using structured light projection has the characteristics of non-contact,high precision,easy operation,and strong real-time performance.However,for actual measurement,projection modulated images are disturbed by electronic noise or other interference,which reduces the precision of the measurement system.To solve this problem,a 3D measurement algorithm of structured light based on deep learning is proposed.The end-to-end multi-convolution neural network model is designed to separately extract the coarse-and fine-layer features of a 3D image.The point-cloud model is obtained by nonlinear regression.The weighting coefficient loss function is introduced to the multi-convolution neural network,and the point-cloud data are continuously optimized to obtain the 3D reconstruction model.To verify the effectiveness of the method,image datasets of different 3D gypsum models were collected,trained,and tested using the above method.Experimental results show that the algorithm effectively eliminates external light environmental interference,avoids the influence of object shape,and achieves higher stability and precision.The proposed method is proved to be effective for regular objects.

MULTIPARAMETER MEASUREMENT FOR RACEWAY GROOVE OF BEARING BASED ON 3D RECONSTRUCTION WITH DIGITAL STRUCTURED LIGHT

A fast 3D reconstruction method based on structured light to measure various parameters of the raceway groove is presented. Digital parallel grating stripes distributed with sine density are projected onto the raceway groove by a DLP projector, and distorting of stripes is happened on the raceway. Simultaneously, aided by three-step phase-shifting approach, three images covered by different stripes are obtained by a high-resolution CCD camera at the same location, thus a more accuracy local topography can be obtained. And then the bearing is rotated on a high precision computer controlled rotational stage. Three images are also obtained as the former step at next planned location triggered by the motor. After one cycle, all images information is combined through the mosaics. As a result, the 3D information of raceway groove can be gained. Not only geometric properties but also surface flaws can be extracted by software. A preliminary hardware system has been built, with which some geometric parameters have been extracted from reconstructed local topography.

Calibration of line structured light vision system based on camera’s projective center

Based on the characteristics of line structured light sensor, a speedy method for the calibration was established. With the coplanar reference target, the spacial pose between camera and optical plane can be calibrated by using of the camera’s projective center and the light’s information in the camera’s image surface. Without striction to the movement of the coplanar reference target and assistant adjustment equipment, this calibration method can be implemented. This method has been used and decreased the cost of calibration equipment, simplified the calibration procedure, improved calibration efficiency. Using experiment, the sensor can attain relative accuracy about 0.5%, which indicates the rationality and effectivity of this method.

Application of color structured light pattern to measurement of large out-of-plane deformation

Measurement of out-of-plane deformation is significant to understanding of the deflection mechanisms of the plate and tube structures.In this study,a new surface contouring technique with color structured light is applied to measure the out-of-plane deformation of structures with one-shot projection.Through color fringe recognizing,decoding and triangulation processing for the captured images corresponding to each deformation state,the feasibility of the method is testified by the measurement of elastic deflections of a flexible square plate,showing good agreement with those from the calibrated displacement driver.The plastic deformation of two alloy aluminum rectangular tubes is measured to show the technique application to surface topographic evaluation of the buckling structures with large displacements.

Hybrid Visual Servoing Control for Robotic Arc Welding Based on Structured Light Vision

A novel hybrid visual servoing control method based on structured light vision is pro-posed for robotic arc welding with a general six degrees of freedom robot. It consists of a positioncontrol inner-loop in Cartesian space and two outer-loops. One is position-based visual control inCartesian space for moving in the direction of weld seam, i.e., weld seam tracking, another is image-based visual control in image space for adjustment to eliminate the errors in the process of tracking.A new Jacobian matrix from image space of the feature point on structured light stripe to Cartesianspace is provided for dierential movement of the end-e?ector. The control system model is simplifiedand its stability is discussed. An experiment of arc welding protected by gas CO2 for verifying iswell conducted.

Influence of sampling on face measuring system based on composite structured light

Human face can be rebuilt to a three-dimensional (3 D) digital profile based on an optical 3D sensing system named Composite Fourier-Transform Profilometry (CFTP) where a composite structured light will be used. To study the sampling effect during the digitization process in practical CFTP, the pectinate function and convolution theorem were introduced to discuss the potential phase errors caused by sampling the composite pattern along two orthogonal directions. The selecting criterions of sampling frequencies are derived and the results indicate that to avoid spectral aliasing, the sampling frequency along the phrase variation direction must be at least four times as the baseband and along the orthogonal direction it must be at least three times as the larger frequency of the two carrier frequencies. The practical experiment of a model face reconstruction verified the theories.

Filtering algorithm of line structured light for long-distance obstacle detection

Since unmanned ground vehicles often encounter concave and convex obstacles in wild ground, a filtering algorithm using line structured light to detect these long distance obstacles is proposed. For the line structured light image, a ranked-order based adaptively extremum median （RAEM） filter algorithm on salt and pepper noise is presented. In the algorithm, firstly effective points and noise points in a filtering window are differentiated; then the gray values of noise points are replaced by the medium of gray values of the effective pixels, with the efficient points＇ gray values unchanged; in the end this algorithm is proved to be efficient by experiments. Experimental resuits demonstrate that the image blur, resulting into proposed algorithm can remove noise points effectively and minimize the protecting the edge information as much as possible.

Projecting Distortion Calibration and Evaluation of Coding Fringes in Structured Light System

Coded structured light is an accurate, fast 3D measurement approach with high sampling density, of which the encoded fringes are distorted when projected to curved surface. Focused on the demand of encoding, decoding, multiview registration and system calibration, we expect to obtain undistorted fringes from camera image. Therefore, in this paper, we analyze the accuracy and sampling density of projecting distortion calibration approach based on control point and fitting surface. Moreover, combining the characteristic of coded structured light system, we design encoded fringe projecting distortion calibration scheme based on simplified encoded structured light model. Primarily, we neglect the minor parameters that affect the calibration in structured light model to reduce complexity. Then, we build the correspondence between camera image points and projector image points and achieve the calibration. Finally, we design evaluation scheme of projecting distortion calibration with parallelism and equal interval, and verify the effectiveness and accuracy of the approach through visual effect and experimental data.

Calibration for 3D Structured Light Measurement

A calibration procedure was developed for three-dimensional(3D) binocular structured light measurement systems. In virtue of a specially designed pattern, matching points in stereo images are extracted. And then sufficient 3D space points are obtained through pairs of images with the intrinsic and extrinsic parameters of each camera estimated prior and consequently some lights are calibrated by means of multi point fitting. Finally, a mathematical model is applied to interpolate and approximate all dynamic scanning lights based on geometry. The process of calibration method is successfully used in the binocular 3D measurement system based on structured lights and the 3D reconstruction results are satisfying.

Structured-light Image Compression Based on Fractal Theory

The method of fractal image compression is introduced which is applied to compress the line structured light image. Based on the self similarity of the structured light image, we attain satisfactory compression ratio and higher peak signal to noise ratio (PSNR). The experimental results indicate that this method can achieve high performance.