Research and development of fringe projection-based methods in 3D shape reconstruction

This paper discusses current research and development of fringe projection-based techniques. A system based on Fourier transform profilometry (FTP) is proposed for three-dimensional (3D) shape recovery. The system improves the method of phase unwrapping to gain accurate 3D shapes of objects. The method uses a region-growing algorithm for the path prediction guided by the quality map to increase the recovering accuracy and provides a fast and simple tool for 3D shape recovery. The shape measurement and data recovery are integrated to offer a new method of 3D modelling. Examples are presented to verify the feasibility of the proposed method.

Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement

Single-shot high-speed 3D imaging is important for reconstructions of dynamic objects.For fringe projection profilometry(FPP),however,it is still challenging to recover accurate 3D shapes of isolated objects by a single fringe image.In this paper,we demonstrate that the deep neural networks can be trained to directly recover the absolute phase from a unique fringe image that involves spatially multiplexed fringe patterns of different frequencies.The extracted phase is free from spectrum-aliasing problem which is hard to avoid for traditional spatial-multiplexing methods.Experiments on both static and dynamic scenes show that the proposed approach is robust to object motion and can obtain high-quality 3D reconstructions of isolated objects within a single fringe image.

A convenient look-up-table based method for the compensation of non-linear error in digital fringe projection

Although the structured light system that uses digital fringe projection has been widely implemented in three-dimensional surface profile measurement, the measurement system is susceptible to non-linear error. In this work, we propose a convenient look-up-table-based （LUT-based） method to compensate for the non-linear error in captured fringe patterns. Without extra calibration, this LUT-based method completely utilizes the captured fringe pattern by recording the full-field differences. Then, a phase compensation map is established to revise the measured phase. Experimental results demonstrate that this method works effectively.

Fringe Projection Measurement System in Reverse Engineering

Acquisition of physical data with high precision is a key step in reverse engineering (RE). It is an important stimulative for the progress of reverse engineering with which various digitizing devices are invent ed, developed and made applicable. This paper introduces a three dimensional opt ical measurement method based on digital fringe projection technique in RE to im prove the technique through its application. A practical example is presented an d the result demonstrates the applicability and feasibility of the measurement s ystem as well as the reliability and validity of relevant methods and algorithms .

Measuring the kinematics of a free-flying hawk-moth(Macroglossum stellatarum)by a comb-fringe projection method

We describe a 2D fringe projection method that involves projecting two groups of sharp comb fringes onto a free-flying hawk-moth from different directions and recording the images of distorted fringes by two high speed cameras from two orthogonal views. By calculating the 3D coordinates of the points on the hawk-moth and three-dimensional reconstruction of the wing, the flight trajectory, body attitude and wing kinematics including flapping angle, elevation angle, torsion angle, and camber deformation are obtained.

Quality Guided Phase Unwrapping for Fringe Projection Profilometry

Fringe projection technique is a non-contact, full field 3-D shape measurement method. The object depth information is recorded in one or several deformed fringe patterns. The phase-shifting algorithm or the Fourier transform method can be used to extract the wrapped phase data. A phase unwrapping process is then applied to retrieve a continuous phase distribution, which represents the surface profile of the test object. In this paper, a quality-guided phase unwrapping approach is incorporated and two novel phase quality evaluation methods are proposed to facilitate the phase unwrapping process.

Optical/Digital Fringe Multiplication in Projection Moire

In this paper, the optical/digital fringe multiplication (O/DFM) is realized in projection moiré using digital image processing (DIP). Both deformed grating and reference grating are captured using a CCD camera. The reference grating can also be produced using DIP. With the O/DFM, the pattern is multiplied with an image processing software, which is developed using MATLAB 6.5. Also with DIP, the phase shifting can become much simpler, and the automation can be adopted. The multiplied pattern is much thinner and easier to read, and sensitivity of measurement can be enhanced.

A study of making use of computer-aided projection fringe system to measure breast dimension

To measure breast basic dimension by using computer-aided projection fringe system.Methods A system has been developed for measuring breast basic dimension based on computer-aided projection fringe measurement and programming software.Plastic manikins breast’s SN-N (sternal notch to nipple distance),N-ML (nipple to midline distance),N-N (internipple distance),MBW (base width of breast) and N-IMF (nipple to inframammary fold distance) are measured with this system.At the same time,these items are also measured with routine ruler.Results This study indicate that the system has some merits:① non-touching measurement;② it is very rapid,the patient measured need hold his breath only 0.5 second,and all the time it takes is about 2.5 minutes;③ the measurement’s sensitivity is as high as to 0.6 mm,which meets the clinic requirement entirely;④ the measurement’s accuracy of the system is not significantly when comparing to the routine ruler’s.Conclusion Computer-adided projection fringe system for measuring breast basic dimension is feasible and advanced.14 refs,1 fig.

Systematic Radio Telescope Alignment Using Portable Fringe Projection Profilometry

In 2019,the Event Horizon Telescope(EHT)released the first-ever image of a black hole event horizon.Astronomers are now aiming for higher angular resolutions of distant targets,like black holes,to understand more about the fundamental laws of gravity that govern our universe.To achieve this higher resolution and increased sensitivity,larger radio telescopes are needed to operate at higher frequencies and in larger quantities.Projects like the next-generation Very Large Array(ngVLA)and the Square-Kilometer Array(SKA)require building hundreds of telescopes with diameters greater than 10 ms over the next decade.This has a twofold effect.Radio telescope surfaces need to be more accurate to operate at higher frequencies,and the logistics involved in maintaining a radio telescope need to be simplified to support them properly in large quantities.Both of these problems can be solved with improved methods for surface metrology that are faster and more accurate with a higher resolution.This leads to faster and more accurate panel alignment and,therefore,a more productive observatory.In this paper,we present the use of binocular fringe projection profilometry as a solution to this problem and demonstrate it by aligning two panels on a 3-m radio telescope dish.The measurement takes only 10 min and directly delivers feedback on the tip,tilt,and piston of each panel to create the ideal reflector shape.

Projecting Distortion Calibration and Evaluation of Coding Fringes in Structured Light System

Coded structured light is an accurate, fast 3D measurement approach with high sampling density, of which the encoded fringes are distorted when projected to curved surface. Focused on the demand of encoding, decoding, multiview registration and system calibration, we expect to obtain undistorted fringes from camera image. Therefore, in this paper, we analyze the accuracy and sampling density of projecting distortion calibration approach based on control point and fitting surface. Moreover, combining the characteristic of coded structured light system, we design encoded fringe projecting distortion calibration scheme based on simplified encoded structured light model. Primarily, we neglect the minor parameters that affect the calibration in structured light model to reduce complexity. Then, we build the correspondence between camera image points and projector image points and achieve the calibration. Finally, we design evaluation scheme of projecting distortion calibration with parallelism and equal interval, and verify the effectiveness and accuracy of the approach through visual effect and experimental data.

Phase unwrapping based on deep learning in light field fringe projection 3D measurement

Phase unwrapping is one of the key roles in fringe projection three-dimensional(3D)measurement technology.We propose a new method to achieve phase unwrapping in camera array light filed fringe projection 3D measurement based on deep learning.A multi-stream convolutional neural network(CNN)is proposed to learn the mapping relationship between camera array light filed wrapped phases and fringe orders of the expected central view,and is used to predict the fringe order to achieve the phase unwrapping.Experiments are performed on the light field fringe projection data generated by the simulated camera array fringe projection measurement system in Blender and by the experimental 3×3 camera array light field fringe projection system.The performance of the proposed network with light field wrapped phases using multiple directions as network input data is studied,and the advantages of phase unwrapping based on deep learning in light filed fringe projection are demonstrated.

彩色光栅投影的三维轮廓自动化测量研究

为了测量物体的三维轮廓数据,提出一种基于改进彩色光栅投影三维轮廓测量技术,并利用DLP来投影彩色光栅条纹到被测物体上来测量被测物体的高度,由于DLP的投射范围太大,无法直接投射在微小物体上,因此通过自行改装的方式改变DLP的投射范围,将其投影面积限制在约20×20 mm内,使其能覆盖整个被测物体。但由于改装时所选用的DLP镜头并不是针对DLP而设计的,因此容易产生光场分布不均的情况,可对无光栅图像或有彩色光栅图像进行光场补偿,使光栅图像更为清楚,用于对被测物体进行更准确的三维轮廓测量与重建分析计算,且获取被测物体的三维数据精度可达2μm,速度在0.024 s内。实验结果表明,该方法具有更高的速度、灵敏度及适用性。

基于条纹投影三目系统的舌面完整点云获取方法

传统的舌诊过程存在客观性较差的问题,将计算机视觉技术应用于舌诊可以有效推进舌诊客观化。双目立体视觉结合结构光条纹投影技术是三维测量的常用手段,但对于舌诊的测量场景,由于舌体表面常带有津液和口水等,造成条纹图像出现分布随机性较大的高反光区域,导致相位解算错误和点云缺失。本文提出了一种基于光栅投影的三目测量系统,由3个相机构成1个三目和3个双目子系统。在相位和级次约束基础上引入双极线约束,提高三目立体匹配精度,并利用匹配点补充优化三目匹配点集,重建高反光区域点云。结果显示,对比传统双目系统,本系统的匹配精度和重建精度均得到提升,特别是在真实舌面测量中能够生成纹理清晰、特征完整的点云,对舌面高反光区域进行了有效测量,有助于舌诊的客观化进行。

基于Hilbert变换的条纹-灰度线性投影三维测量

针对传统的条纹投影测量中绝对相位求解方法存在投影图像数量较多以及算法复杂程度高的问题,提出了一种基于Hilbert变换的条纹-灰度线性投影策略及相位求解方法,该方法仅使用3幅投影图像——1幅高频条纹图和2幅灰度线性变化图,通过Hilbert变换求得包裹相位,采用灰度线性变化图像得到基础相位,联立包裹相位和基础相位可得到条纹级数,进而得到绝对相位。通过构建的主动双目系统得到的三维重构结果发现,该方法可以有效恢复被测物体的三维形状,测量精度可达0.1 mm。

高动态范围条纹结构光在机检测技术及应用进展

条纹结构光技术是近年来发展迅速的非接触式测量方法,为机械加工在机检测提供了新的解决方案。由于加工环境光线复杂且金属零件本身具有高反光的特性,造成结构光在机检测的精度降低。将高动态范围(High Dynamic Range,HDR)技术应用于结构光检测中,可抑制高反光的影响,实现金属零件在复杂场景的测量。本文首先介绍了结构光测量原理,总结出HDR结构光在机检测面临的难点;其次,对HDR结构光技术进行了全面综述,以机械加工在机检测为背景,对基于硬件设备的HDR技术和基于条纹算法的HDR技术分别进行了归纳分析;然后,根据在机检测的条件需求,对各类技术进行总结,并比较不同方法的优缺点和在机检测的适用性;最后,结合近年来先进制造技术和精密测量的研究热点,对潜在应用进行分析,提出技术展望。

基于条纹投影的复杂结构多维度信息传感技术(内封面文章·特邀)

随着精密加工和智能制造业的高速发展,在航空航天、汽车工业、工业检测等领域中,对复杂结构,例如:发动机组件、蜂窝结构件等在生产制造与使用过程中的形貌、形变及应变等多维度信息的测量需求日益增加。最近兴起的数字图像相关辅助条纹投影技术已被证明在复杂结构的形貌和形变测量方面具有独特优势。但目前由于两类方法各自不同的技术特点,基于两者结合的形貌、形变测量方法在测量精度、效率等方面存在诸多矛盾,难以兼顾高分辨率、多维度的测量需求,阻碍了该类技术进一步拓展应用。综述了课题组近年来发展的基于条纹投影的复杂结构多维度信息传感技术及最新的研究进展,系统回顾了两类方法结合时的条纹与散斑信息干扰、原始纹理被破坏、编码效率低等技术难点,并总结了对应的解决方案,最终实现了对复合材料编织结构、颗粒膨胀减震结构以及层叠结构等复杂结构的多维度信息传感。文中所提方法在不增加额外硬件的前提下,将传统条纹投影测量系统升级为多维度信息传感系统,可以同时测量2D纹理(T)、4D形貌(x,y,z,t)以及分析维度力学参数(形变d和应变s),突破了复杂结构高分辨率、高精度纹理形貌重建与形变应变分析的技术壁垒,有望在复杂结构可靠性分析、复合材料性能评估等领域中得到应用。

面向高反物体的区域映射自适应条纹投影方法

对于具有高反射率表面物体的测量,传统条纹投影方法无法测量出完整的三维信息。为了解决这一问题,采用了区域映射条纹投影方法(RMAFP),通过图像灰度值区分饱和区域,再采用改进的相移算法,得到饱和区域的相位与饱和度;相位值实现投影区域映射,再投影根据饱和度修正的标准条纹图,对原饱和区域重新测量,即可消除饱和现象,结合原非饱和区域像素的高精度相位,就能完成对高反物体的准确测量。实验过程仅需两轮投影即可实现,实验结果显示高反点的测量准确率为98%,同时对所有高反点均能得到较为准确的测量结果,表明该方法切实可行,可以完整地对复杂高反射率表面进行测量。

基于条纹投影轮廓术的航空高光零件三维测量技术

针对航空工业零件高光表面结构光三维测量中强反光和多次反光问题,提出了基于高动态多步相移和单像素成像相结合的三维测量技术,有效地解决了测量复杂反光表面时的点云缺失和精度下降问题。对标准量块进行三维测量和精度评价,基于高动态多步相移的三维测量技术拟合均方根误差为0.0066 mm,基于单像素成像的三维测量技术拟合均方根误差为0.0046 mm。以台阶标准器作为测量目标,开展绝对尺寸测量精度验证试验,绝对偏差不大于0.007 mm。分别对金属表面结构件(如航空发动机叶轮、涡轮叶片)和碳纤维结构件进行测量试验。试验结果表明,三维测量结果基本无点云缺失,强反光和多次反光区域均获取了完整的点云数据,实现了航空工业高光零件的精密三维测量。

基于相位立体匹配的金属断口三维形貌测量

为分析金属材料的力学性能,对金属断口的形貌特征进行分析来评定金属性能。为准确获取金属断口的三维形貌,设计了一种基于相位立体匹配的金属断口三维形貌测量系统,通过相机采集投射到金属断口表面的条纹,结合多频相移外差算法解算出断口表面相位信息,采用双目立体匹配获取断口表面三维形貌。相位立体匹配算法能解决金属断口弱纹理区域匹配精度不高、点云信息丢失的问题。金属量块测量实验显示,系统精度能达到分析金属材料性能的要求,通过对脆性和塑性材料的扭曲、弯曲断裂等金属试样的测量,得到了金属断口的三维形貌数据。实验显示,所提方法精度在0.05 mm内且保证了在金属断口弱纹理区域的匹配精度和密集的点云信息,在金属材料的力学特性分析中有很好的应用前景。