Generation of structured light beams with polarization variation along arbitrary spatial trajectories using tri-layer metasurfaces

Conventionally,the spatially structured light beams produced by metasurfaces primarily highlight the polarization modulation of the beams propagating along the optical axis or the beams'spatial transmission trajectory.In particular,along the optical axis,the polarization state is either constant or varies continuously in each output plane.Here,we develop innovative spatially structured light beams with continually changing polarization along any arbitrary spatial transmission trajectories.With tri-layer metallic metasurfaces,the geometric characteristics of each layer structure can be adjusted to modulate the phase and polarization state of the incident terahertz(THz)wave.The beam will converge to the predefined trajectory along several paths to generate a Bessel-like beam with longitudinal polarization changes.We demonstrate the versatility of the approach by designing two THz-band structured light beams with varying polarization states along the spatial helical transmission trajectory.Continuous linear polarization changes and linear polarization to right circular polarization(RCP)and back to linear polarization changes are realized respectively.The experimental results are basically consistent with the simulated results.Our proposal for arbitrary trajectory structured light beams with longitudinally varying polarization offers a practical method for continuously regulating the characteristics of spatial structured light beams with non-axial transmission.This technique has potential uses in optical encryption,particle manipulation,and biomedical imaging.

Nonlinear optics with structured light

The interest in tailoring light in all its degrees of freedom is steadily gaining traction,driven by the tremendous developments in the toolkit for the creation,control and detection of what is now called structured light.Because the complexity of these optical fields is generally understood in terms of interference,the tools have historically been linear optical elements that create the desired superpositions.For this reason,despite the long and impressive history of nonlinear optics,only recently has the spatial structure of light in nonlinear processes come to the fore.In this review we provide a concise theoretical framework for understanding nonlinear optics in the context of structured light,offering an overview and perspective on the progress made,and the challenges that remain.

Seam Tracking and Visual Control for Robotic Arc Welding Based on Structured Light Stereovision

A real-time arc welding robot visual control system based on a local network with a multi-level hierarchy is developed in this paper. It consists of an intelligence and human-machine interface level, a motion planning level, a motion control level and a servo control level. The last three levels form a local real-time open robot controller, which realizes motion planning and motion control of a robot. A camera calibration method based on the relative movement of the end-effector connected to a robot is proposed and a method for tracking weld seam based on the structured light stereovision is provided. Combining the parameters of the cameras and laser plane, three groups of position values in Cartesian space are obtained for each feature point in a stripe projected on the weld seam. The accurate three-dimensional position of the edge points in the weld seam can be calculated from the obtained parameters with an information fusion algorithm. By calculating the weld seam parameter from position and image data, the movement parameters of the robot used for tracking can be determined. A swing welding experiment of type V groove weld is successfully conducted, the results of which show that the system has high resolution seam tracking in real-time, and works stably and efficiently.

Nonlinear effect of the structured light profilometry in the phase-shifting method and error correction

Digital structured light （SL） profilometry is increasingly used in three-dimensional （3D） measurement technology. However, the nonlinearity of the off-the-shelf projectors and cameras seriously reduces the measurement accuracy. In this paper, first, we review the nonlinear effects of the projector-camera system in the phase-shifting structured light depth measurement method. We show that high order harmonic wave components lead to phase error in the phase-shifting method. Then a practical method based on frequency domain filtering is proposed for nonlinear error reduction. By using this method, the nonlinear calibration of the SL system is not required. Moreover, both the nonlinear effects of the projector and the camera can be effectively reduced. The simulations and experiments have verified our nonlinear correction method.

Calibration method for multi-line structured light vision sensor based on Plücker line

For the rapid calibration of multi-line structured light system,a method based on Plücker line was proposed.Most of the conventional line-structured light calibration methods extract the feature points and transform the coordinates of points to obtain the plane equation.However,a large number of points lead to complicated operation which is not suitable for the application scenarios of multi-line structured light.To solve this issue,a new calibration method was proposed that applied the form of Plücker matrix throughout the whole calibration process,instead of using the point characteristics directly.The advantage of this method is that the light plane equation can be obtained quickly and accurately in the camera coordinate frame.Correspondingly a planar target particularly for calibrating multi-line structured light was also designed.The regular lines were transformed into Plücker lines by extending the two-dimensional image plane and defining a new image space.To transform the coordinate frame of Plücker lines,the perspective projection mathematical model was re-expressed based on the Plücker matrix.According to the properties of the line and plane in the Plücker space,a linear matrix equation was efficiently constructed by combining the Plücker matrices of several coplanar lines so that the line-structured light plane equation could be furtherly solved.The experiments performed validate the proposed method and demonstrate the significant improvement in the calibration accuracy,when the test distance is 1.8 m,the root mean square(RMS)error of the three-dimensional point is within 0.08 mm.

On-line detection system of weld-based rapid prototyping process based on structured light

In order to realize the closed-loop control for rapid prototyping process based on gas metal arc welding, the geometric parameters of weld beads should be detected. In this study, a vision sensor system consisting of a linear laser projector and a CCD camera was designed to collect images of weld beads. Then, an image processing approach which combines with a Gaassian filter and an improved gravity method was used to extract the centerline of a light stripe based on VC ＋＋. Feature points of the centerline were identical directly by means of an image fusion algorithm. Experimental results show that image fusion is an effective approach to measure the width and height of the weld bead with high accuracy. This method can identify beads effectively in multi-pass welding and avoid designing different modes to suit all kinds of shapes.

Three-Dimensional Measurement Using Structured Light Based on Deep Learning

Three-dimensional(3D)reconstruction using structured light projection has the characteristics of non-contact,high precision,easy operation,and strong real-time performance.However,for actual measurement,projection modulated images are disturbed by electronic noise or other interference,which reduces the precision of the measurement system.To solve this problem,a 3D measurement algorithm of structured light based on deep learning is proposed.The end-to-end multi-convolution neural network model is designed to separately extract the coarse-and fine-layer features of a 3D image.The point-cloud model is obtained by nonlinear regression.The weighting coefficient loss function is introduced to the multi-convolution neural network,and the point-cloud data are continuously optimized to obtain the 3D reconstruction model.To verify the effectiveness of the method,image datasets of different 3D gypsum models were collected,trained,and tested using the above method.Experimental results show that the algorithm effectively eliminates external light environmental interference,avoids the influence of object shape,and achieves higher stability and precision.The proposed method is proved to be effective for regular objects.

3D surface reconstruction based on binocular vision using structured light

A 3D surface reconstruction method using a binocular stereo vision technology and a coded structured light,which combines a gray code with phase-shift has been studied.The accuracy of the 3 D surface reconstruction mainly depends on the decoding of gray code views and phase-shift views.In order to find the boundary accurately,gray code patterns and their inverses are projected onto a human eye plaster model.The period dislocation between the gray code views and the phase-shift views in the course of decoding has been analyzed and a new method has been proposed to solve it.The splicing method is based on feature points.The result of the 3D surface reconstruction shows the accuracy and reliability of our method.

Construction for M-arrays and application in structured light

M-arrays are random arrays in which an appropriate sub-window appears only once in the whole array. Coded structured light based on M-arrays is one=shot technique to rapidly acquire 3D information of unknown surfaces by projecting suitable patterns onto a measuring surface. This paper presents a method to construct large size M-arrays based on the piece growing algorithm in which an array is constructed by many pieces through splicing each other. Reconstructing 3D shapes by utilizing the designed pattern based on constructed M-arrays for two objects are given.

SEGMENTATION ALGORITHM BASED ON EDGE-SEARCHING FOR MULTI-LINEAR STRUCTURED LIGHT IMAGES

Aiming at the problem that the existence of disturbances on the edges of light-stripe makes the segmentation of the light-stripes images difficult, a new segmentation algorithm based on edge-searching is presented. It firstly calculates every edge pixel＇s horizontal coordinate grads to produce the corresponding grads-edge, then uses a designed length-variable l D template to scan the light-stripes＇ grads-edges. The template is able to find the disturbances with different width utilizing the distributing character of the edge disturbances. The found disturbances are eliminated finally. The algorithm not only can smoothly segment the light-stripes images, but also eliminate most disturbances on the light-stripes＇ edges without damaging the light-stripes images＇ 3D information. A practical example of using the proposed algorithm is given in the end. It is proved that the efficiency of the algorithm has been improved obviously by comparison.

Research on Visual Detection Algorithm for Groove Feature Sizes by Means of Structured Light Projection

The visual inspection is an economical and effective method for welding. For measuring the feature sizes of grooves,a method based on line structured light is presented. Firstly,an adaptive algorithm to extract the subpixel centerline of structured light stripes is introduced to deal with the uneven width and grayscale distributions of laser stripes,which is based on the quadratic weighted grayscale centroid. By means of region-of-interest(ROI)division and image difference,an image preprocessing algorithm is developed for filtering noise and improving image quality. Furthermore,to acquire geometrical dimensions of various grooves and groove types precisely,the subpixel feature point extraction algorithm of grooves is designed. Finally, experimental results of feature size measuring show that the absolute error of measurement is 0.031—0.176 mm,and the relative error of measurement is 0.2%—3.6%.

MULTIPARAMETER MEASUREMENT FOR RACEWAY GROOVE OF BEARING BASED ON 3D RECONSTRUCTION WITH DIGITAL STRUCTURED LIGHT

A fast 3D reconstruction method based on structured light to measure various parameters of the raceway groove is presented. Digital parallel grating stripes distributed with sine density are projected onto the raceway groove by a DLP projector, and distorting of stripes is happened on the raceway. Simultaneously, aided by three-step phase-shifting approach, three images covered by different stripes are obtained by a high-resolution CCD camera at the same location, thus a more accuracy local topography can be obtained. And then the bearing is rotated on a high precision computer controlled rotational stage. Three images are also obtained as the former step at next planned location triggered by the motor. After one cycle, all images information is combined through the mosaics. As a result, the 3D information of raceway groove can be gained. Not only geometric properties but also surface flaws can be extracted by software. A preliminary hardware system has been built, with which some geometric parameters have been extracted from reconstructed local topography.

Application of color structured light pattern to measurement of large out-of-plane deformation

Measurement of out-of-plane deformation is significant to understanding of the deflection mechanisms of the plate and tube structures.In this study,a new surface contouring technique with color structured light is applied to measure the out-of-plane deformation of structures with one-shot projection.Through color fringe recognizing,decoding and triangulation processing for the captured images corresponding to each deformation state,the feasibility of the method is testified by the measurement of elastic deflections of a flexible square plate,showing good agreement with those from the calibrated displacement driver.The plastic deformation of two alloy aluminum rectangular tubes is measured to show the technique application to surface topographic evaluation of the buckling structures with large displacements.

Filtering algorithm of line structured light for long-distance obstacle detection

Since unmanned ground vehicles often encounter concave and convex obstacles in wild ground, a filtering algorithm using line structured light to detect these long distance obstacles is proposed. For the line structured light image, a ranked-order based adaptively extremum median （RAEM） filter algorithm on salt and pepper noise is presented. In the algorithm, firstly effective points and noise points in a filtering window are differentiated; then the gray values of noise points are replaced by the medium of gray values of the effective pixels, with the efficient points＇ gray values unchanged; in the end this algorithm is proved to be efficient by experiments. Experimental resuits demonstrate that the image blur, resulting into proposed algorithm can remove noise points effectively and minimize the protecting the edge information as much as possible.

Calibration for 3D Structured Light Measurement

A calibration procedure was developed for three-dimensional(3D) binocular structured light measurement systems. In virtue of a specially designed pattern, matching points in stereo images are extracted. And then sufficient 3D space points are obtained through pairs of images with the intrinsic and extrinsic parameters of each camera estimated prior and consequently some lights are calibrated by means of multi point fitting. Finally, a mathematical model is applied to interpolate and approximate all dynamic scanning lights based on geometry. The process of calibration method is successfully used in the binocular 3D measurement system based on structured lights and the 3D reconstruction results are satisfying.

THREE DIMENSIONAL DIGITIZATION OF HUMAN HEAD BY FUSING STRUCTURED LIGHT AND CONTOURS

Three dimensional digitization of human head is desired in many applications. In this paper, an information fusion based scheme is presented to obtain 3-D information of human head. Structured light technology is employed to measure depth. For the special reflection areas,in which the structured light stripe can not be detected directly, the shape of the structured light stripe can be calculated from the corresponding contour. By fusing the information of structured light and the contours, the problem of reflectance influence is solved, and the whole shape of head,including hair area, can be obtained. Some good results are obtained.

Generative deep-learning-embedded asynchronous structured light for three-dimensional imaging

Three-dimensional(3D)imaging with structured light is crucial in diverse scenarios,ranging from intelligent manufacturing and medicine to entertainment.However,current structured light methods rely on projector-camera synchronization,limiting the use of affordable imaging devices and their consumer applications.In this work,we introduce an asynchronous structured light imaging approach based on generative deep neural networks to relax the synchronization constraint,accomplishing the challenges of fringe pattern aliasing,without relying on any a priori constraint of the projection system.To overcome this need,we propose a generative deep neural network with U-Net-like encoder-decoder architecture to learn the underlying fringe features directly by exploring the intrinsic prior principles in the fringe pattern aliasing.We train within an adversarial learning framework and supervise the network training via a statisticsinformed loss function.We demonstrate that by evaluating the performance on fields of intensity,phase,and 3D reconstruction.It is shown that the trained network can separate aliased fringe patterns for producing comparable results with the synchronous one:the absolute error is no greater than 8μm,and the standard deviation does not exceed 3μm.Evaluation results on multiple objects and pattern types show it could be generalized for any asynchronous structured light scene.

Integrated structured light manipulation

Structured light,also known as tailored light,shaped light,sculpted light,or custom light,refers to a series of special light beams with spatially variant amplitudes and phases,polarization distributions,or more general spatiotemporal profiles.In the past decades,structured light featuring distinct properties and unique spatial or spatiotemporal structures has grown into a significant research field and given rise to many developments from fundamentals to applications.Very recently,integrated structured light manipulation has become an important trend in the frontier of light field manipulation and attracted increasing interest as a highly promising technique for shaping structured light in an integrated,compact,and miniaturized manner.In this article,we give a comprehensive overview of recent advances in integrated structured light manipulation(generation,processing,detection,and application).After briefly introducing the basic concept and development history of structured light,we present representative works in four important aspects of integrated structured light manipulation,including multiple types of integrated structured light generation,many sorts of integrated structured light processing,diverse forms of integrated structured light detection,and various kinds of integrated structured light applications.We focus on summarizing the progress of integrated structured light manipulation from basic theories to cutting-edge technologies,to key devices,and to a wide variety of applications,from orbital angular momentum carrying light beams to more general structured light beams,from passive to active integration platforms,from micro-nano structures and metasurfaces to 2D photonic integrated circuits and 3D photonic chips,from in-plane to out-of-plane,from multiplexing to transformation,from linear to nonlinear,from classical to quantum,from optical communications to optical holography,imaging,microscopy,trapping,tweezers,metrology,etc.Finally,we also discuss in detail the future trends,opportunities,challenges,and solutions,and give a vision for integrated structured light manipulation.

Robust structured light in atmospheric turbulence

Structured light is routinely used in free-space optical communication channels,both classical and quantum,where information is encoded in the spatial structure of the mode for increased bandwidth.Both real-world and experimentally simulated turbulence conditions have revealed that free-space structured light modes are perturbed in some manner by turbulence,resulting in both amplitude and phase distortions,and consequently,much attention has focused on whether one mode type is more robust than another,but with seemingly inconclusive and contradictory results.We present complex forms of structured light that are invariant under propagation through the atmosphere:the true eigenmodes of atmospheric turbulence.We provide a theoretical procedure for obtaining these eigenmodes and confirm their invariance both numerically and experimentally.Although we have demonstrated the approach on atmospheric turbulence,its generality allows it to be extended to other channels too,such as aberrated paths,underwater,and in optical fiber.

High Curvature Stripe Profile Extraction Algorithm of Line Structured Light Measuring System

In the line structured light measuring system, the accuracy of the process of laser stripe directly affects the measurement results. Therefore, the extraction algorithm for the laser stripe, especially the surface with high reflection and high curvature, is very important. The imaging principle of line structured light, the light intensity distribution law of laser stripe and the extraction algorithm have been studied, and a stripe profile extraction method based on real light intensity distribution has been proposed. In this algorithm, fast region of interest extraction, stripe width estimation, and adaptive filtering on the striped image are performed. Then the energy center of the stripe at the sub-pixel level is extracted. Finally, the low-quality center points are eliminated, and the context information is used to recover the missing central points. Simulated images generated based on the imaging principle of line structured light and real experimental images were used to evaluate the accuracy and repeatability of the proposed method. The results show that the method behaves excellently at the edges of high-curvature stripes;the maximum error is only 1.6 pixels, which is 1/10 of the classic Steger algorithm;the experiment repeatability is only 8.8 μm, which is 2.7 times that of the Steger method. Therefore, the proposed method improves the accuracy of object contour extraction, and it is especially suitable for contour detection of objects with high curvature.