MULTIPARAMETER MEASUREMENT FOR RACEWAY GROOVE OF BEARING BASED ON 3D RECONSTRUCTION WITH DIGITAL STRUCTURED LIGHT

A fast 3D reconstruction method based on structured light to measure various parameters of the raceway groove is presented. Digital parallel grating stripes distributed with sine density are projected onto the raceway groove by a DLP projector, and distorting of stripes is happened on the raceway. Simultaneously, aided by three-step phase-shifting approach, three images covered by different stripes are obtained by a high-resolution CCD camera at the same location, thus a more accuracy local topography can be obtained. And then the bearing is rotated on a high precision computer controlled rotational stage. Three images are also obtained as the former step at next planned location triggered by the motor. After one cycle, all images information is combined through the mosaics. As a result, the 3D information of raceway groove can be gained. Not only geometric properties but also surface flaws can be extracted by software. A preliminary hardware system has been built, with which some geometric parameters have been extracted from reconstructed local topography.

3D reconstruction of worn parts for flexible remanufacture based on robotic arc welding

3D reconstruction of worn parts is the foundation for remanufacturing system based on robotic arc welding, because it can provide 3D geometric information for robot task plan. In this investigation, a novel 3D reconstruction system based on linear structured light vision sensing is developed. This system hardware consists of a MTC368-CB CCD camera, a MLH-645 laser projector and a DH-CG300 image grabbing card. This system software is developed to control the image data capture. In order to reconstruct the 3D geometric information from the captured image, a two steps rapid calibration algorithm is proposed. The 3D reconstruction experiment shows a satisfactory result.

Three-Dimensional Measurement Using Structured Light Based on Deep Learning

Three-dimensional(3D)reconstruction using structured light projection has the characteristics of non-contact,high precision,easy operation,and strong real-time performance.However,for actual measurement,projection modulated images are disturbed by electronic noise or other interference,which reduces the precision of the measurement system.To solve this problem,a 3D measurement algorithm of structured light based on deep learning is proposed.The end-to-end multi-convolution neural network model is designed to separately extract the coarse-and fine-layer features of a 3D image.The point-cloud model is obtained by nonlinear regression.The weighting coefficient loss function is introduced to the multi-convolution neural network,and the point-cloud data are continuously optimized to obtain the 3D reconstruction model.To verify the effectiveness of the method,image datasets of different 3D gypsum models were collected,trained,and tested using the above method.Experimental results show that the algorithm effectively eliminates external light environmental interference,avoids the influence of object shape,and achieves higher stability and precision.The proposed method is proved to be effective for regular objects.

Deep-Learning-Empowered 3D Reconstruction for Dehazed Images in IoT-Enhanced Smart Cities

With increasingly more smart cameras deployed in infrastructure and commercial buildings,3D reconstruction can quickly obtain cities’information and improve the efficiency of government services.Images collected in outdoor hazy environments are prone to color distortion and low contrast;thus,the desired visual effect cannot be achieved and the difficulty of target detection is increased.Artificial intelligence(AI)solutions provide great help for dehazy images,which can automatically identify patterns or monitor the environment.Therefore,we propose a 3D reconstruction method of dehazed images for smart cities based on deep learning.First,we propose a fine transmission image deep convolutional regression network(FT-DCRN)dehazing algorithm that uses fine transmission image and atmospheric light value to compute dehazed image.The DCRN is used to obtain the coarse transmission image,which can not only expand the receptive field of the network but also retain the features to maintain the nonlinearity of the overall network.The fine transmission image is obtained by refining the coarse transmission image using a guided filter.The atmospheric light value is estimated according to the position and brightness of the pixels in the original hazy image.Second,we use the dehazed images generated by the FT-DCRN dehazing algorithm for 3D reconstruction.An advanced relaxed iterative fine matching based on the structure from motion(ARI-SFM)algorithm is proposed.The ARISFM algorithm,which obtains the fine matching corner pairs and reduces the number of iterations,establishes an accurate one-to-one matching corner relationship.The experimental results show that our FT-DCRN dehazing algorithm improves the accuracy compared to other representative algorithms.In addition,the ARI-SFM algorithm guarantees the precision and improves the efficiency.

THREE DIMENSIONAL DIGITIZATION OF HUMAN HEAD BY FUSING STRUCTURED LIGHT AND CONTOURS

Three dimensional digitization of human head is desired in many applications. In this paper, an information fusion based scheme is presented to obtain 3-D information of human head. Structured light technology is employed to measure depth. For the special reflection areas,in which the structured light stripe can not be detected directly, the shape of the structured light stripe can be calculated from the corresponding contour. By fusing the information of structured light and the contours, the problem of reflectance influence is solved, and the whole shape of head,including hair area, can be obtained. Some good results are obtained.

Fast 3D reconstruction of dental cast model based on structured light

To quickly obtain accurate 3D data of dental cast model, this paper proposes a 3D reconstruction method for dental cast model based on structured light. This method combines the structured light with the motor turntable to obtain a group of 3D data for the dental cast model from multiple angles, and automatically registers the dental 3D data from multiple angles through the ball calibration of turntable. Compared with the real data of the dental cast model, the maximum error of the 3D reconstruction results in this paper is 0.115 mm. The reconstruction time of this process is about 130s. The experimental results show that the method has high precision and high scanning speed for the 3D reconstruction of the dental cast model.

3D surface reconstruction based on binocular vision using structured light

A 3D surface reconstruction method using a binocular stereo vision technology and a coded structured light,which combines a gray code with phase-shift has been studied.The accuracy of the 3 D surface reconstruction mainly depends on the decoding of gray code views and phase-shift views.In order to find the boundary accurately,gray code patterns and their inverses are projected onto a human eye plaster model.The period dislocation between the gray code views and the phase-shift views in the course of decoding has been analyzed and a new method has been proposed to solve it.The splicing method is based on feature points.The result of the 3D surface reconstruction shows the accuracy and reliability of our method.

Novel SfM-DLT method for metro tunnel 3D reconstruction and Visualization

Metro tunnels play a crucial role in urban transportation.However,with growing tunnel operation periods,defects,and large deformations appearing,these are influencing tunnel structural performance and threatening public safety.Three-dimensional(3D)tunnel reconstruction is an effective way to highlight tunnel conditions and provide a basis for engineering management and maintenance.However,the current methods of tunnel 3D reconstruction do not sufficiently combine the qualitative and quantitative characteristics of tunnel states.In this study,a novel method for metro tunnel 3D reconstruction based on structure from motion(SfM)and direct linear transformation(DLT)is proposed.The dimensionless 3D reconstruction point cloud acquired through the SfM method showcases the qualitative characteristics(such as leakage and pipelines)of the tunnel state.The close-range photogrammetry DLT method provides scale information missing from the SfM method and quantitative characteristics(such as profile deformation)of the tunnel state.The SfM-DLT method was tested in a Shanghai metro tunnel,and proved to be feasible and promising for future tunnel inspections.

Construction for M-arrays and application in structured light

M-arrays are random arrays in which an appropriate sub-window appears only once in the whole array. Coded structured light based on M-arrays is one=shot technique to rapidly acquire 3D information of unknown surfaces by projecting suitable patterns onto a measuring surface. This paper presents a method to construct large size M-arrays based on the piece growing algorithm in which an array is constructed by many pieces through splicing each other. Reconstructing 3D shapes by utilizing the designed pattern based on constructed M-arrays for two objects are given.

Viewpoint Planning for Freeform Surface Inspection Using Plane Structured Light Scanners

This paper proposes an automatic model-based viewpoint planning method, which can achieve high precision and high efficiency for freeform surfaces inspection using plane structured light scanners. The surface model is utilized in stereolithography format, which is widely used as an industrial standard. The proposed method consists of 4 steps： topology reconstruction, mesh refinement, scan direction determination and viewpoint generation. In the first step, the topology structure of the surface model is reconstructed according to a designed data structure, based on which a neighborhood search algorithm is developed. In the second step, big facets in the surface model are segmented into several small ones, which are suitable for viewpoint planning. In the third step, an initial scan region of a viewpoint is grouped by the neighborhood search algorithm combining with total area and normal vector restrictions. Accordingly, the scan direction is determined by the normal vectors of facets in the initial scan region. In the fourth step, the position, the orientation, and the final scan region of the viewpoint are determined by 4 scan constraints, i.e., field of view, working distance range, view angle and overlap. Experimental results verify the effectiveness and advantages of the proposed method.

3D reconstruction and automatic leakage defect quantification of metro tunnel based on SfM-Deep learning method

Various structural defects deteriorate tunnel operation status and threaten public safety.Current tunnel inspection methods face problems of low efficiency,high equipment expense,and difficult data management.Combining the deep learning model and the 3D reconstruction method based on structure from motion(SfM),this paper proposes a novel SfM-Deep learning method for tunnel inspection.The high-quality 3D tunnel model is constructed by using images taken every 1 m along the longitudinal direction.The instance segmentation of leakage in longitudinal images is realized using the mask region-based convolutional neural network deep learning model.The SfM-Deep learning method projects the texture of the images after defect recognition to the 3D model and realizes the visualization of leakage defects.By projecting the model to the design cylindrical surface and expanding it,the tunnel leakage area is quantified.Through its practical application in a Shanghai metro shield tunnel,the reliability of the proposed method was verified.The novel SfM-Deep learning method can help engineers efficiently carry out intelligent tunnel detection.

Single-cell volumetric imaging with light field microscopy: Advances in systems and algorithms

Single-cell volumetric imaging is essential for researching individual characteristics of cells.As a nonscanning imaging technique,lighteld microscopy(LFM)is a critical tool to achieve realtime three-dimensional imaging with the advantage of single-shot.To address the inherent limits including nonuniform resolution and block-wise artifacts,various modied LFM strategies have been developed to provide new insights into the structural and functional information of cells.This review will introduce the principle and development of LFM,discuss the improved approaches based on hardware designs and 3D reconstruction algorithms,and present the applications in single-cell imaging.

Influence of sampling on face measuring system based on composite structured light

Human face can be rebuilt to a three-dimensional (3 D) digital profile based on an optical 3D sensing system named Composite Fourier-Transform Profilometry (CFTP) where a composite structured light will be used. To study the sampling effect during the digitization process in practical CFTP, the pectinate function and convolution theorem were introduced to discuss the potential phase errors caused by sampling the composite pattern along two orthogonal directions. The selecting criterions of sampling frequencies are derived and the results indicate that to avoid spectral aliasing, the sampling frequency along the phrase variation direction must be at least four times as the baseband and along the orthogonal direction it must be at least three times as the larger frequency of the two carrier frequencies. The practical experiment of a model face reconstruction verified the theories.

一种利用平行结构光的3D重构算法

论文提出了一种简便的采用等间距平行结构光的3D重构算法。算法不需要对摄像机进行标定,只需求解参考平面的像到参考平面的单应矩阵和光平面的像到光平面的单应矩阵,利用所求得的单应矩阵便可以从空间物体在结构光照射下的图像求出物体的3D坐标。实验表明算法计算简单、行之有效。

A 3D Measurement Method Based on Coded Image

The binocular stereo vision system is often used to reconstruct 3D point clouds of an object.However,it is challenging to find effective matching points in two object images with similar color or less texture.This will lead to mismatching by using the stereo matching algorithm to calculate the disparity map.In this context,the object can’t be reconstructed precisely.As a countermeasure,this study proposes to combine the Gray code fringe projection with the binocular camera as well as to generate denser point clouds by projecting an active light source to increase the texture of the object,which greatly reduces the reconstruction error caused by the lack of texture.Due to the limitation of the camera viewing angle,a one-perspective binocular camera can only reconstruct the 2.5D model of an object.To obtain the 3D model of an object,point clouds obtained from multiple-view images are processed by coarse registration using the coarse SAC-IA algorithm and fine registration using the ICP algorithm,which is followed by voxel filtering fusion of the point cloud.To improve the reconstruction quality,a polarizer is mounted in front of the cameras to filter out the redundant reflected light.Eventually,the 3D model and the dimension of a vase are obtained after calibration.

Morphological classification and three-dimensional pore structure reconstruction of shale oil reservoirs: A case from the second member of Paleogene Kongdian Formation in the Cangdong Sag, Bohai Bay Basin, East China

This study combines large volume three-dimensional reconstruction via focused ion beam scanning electron microscopy(FIB-SEM) with conventional scanning electron microscope(SEM) observation, automatic mineral identification and characterization system(AMICS) and large-area splicing of SEM images to characterize and classify the microscopic storage space distribution patterns and 3D pore structures of shales in the second member of the Paleogene Kongdian Formation(Kong 2) in the Cangdong Sag of the Bohai Bay Basin. It is shown that:(1) The Kong 2 Member can be divided into seven types according to the distribution patterns of reservoir spaces: felsic shale with intergranular micron pores, felsic shale with intergranular fissures, felsic shale with intergranular pores, hybrid shale with intergranular pores and fissures, hybrid shale with intergranular pores, clay-bearing dolomitic shale with intergranular pores, and clay-free dolomitic shale with intergranular pores.(2) The reservoir of the intergranular fracture type has better storage capacity than that of intergranular pore type. For reservoirs with storage space of intergranular pore type, the dolomitic shale reservoir has the best storage capacity, the hybrid shale comes second, followed by the felsic shale.(3) The felsic shale with intergranular fissures has the best storage capacity and percolation structure, making it the first target in shale oil exploration.(4) The large volume FIB-SEM 3D reconstruction method is able to characterize a large shale volume while maintaining relatively high spatial resolution, and has been demonstrated an effective method in characterizing the 3D storage space in strongly heterogeneous continental shales.

Structured-Light-Field 3D Imaging System With Coaxial Projection

This paper presents a novel 3D measurement method for a light field camera(LFC)in which 3D information of object space is encoded by a microlens array(MLA).The light ray corresponding to each pixel of the LFC is calibrated.Once the matching points from at least two subviews exhibit sub-pixel accuracy,the 3D coordinates can be calculated optimally by intersecting light rays of these points matched through phase coding.Moreover,the proposed method obtains high-resolved results that exceed the subview resolution due to the virtual continuous phase search strategy.Finally,we combine the LFC and coaxial projection to solve the 3D data loss caused by shadowing and occlusion problems.Experimental results verify the feasibility of the proposed method,and the measurement error is about 30μm in a depth range of 60 mm.

Visual perception driven 3D building structure representa tion from airborne laser scanning point cloud

Background Three-dimensional(3D)building models with unambiguous roof plane geometry parameters,roof structure units,and linked topology provide essential data for many applications related to human activities in urban environments.The task of 3D reconstruction from point clouds is still in the development phase,especially the recognition and interpretation of roof topological structures.Methods This study proposes a novel visual perception-based approach to automatically decompose and reconstruct building point clouds into meaningful and simple parametric structures,while the associated mutual relationships between the roof plane geometry and roof structure units are expressed by a hierarchical topology tree.First,a roof plane extraction is performed by a multi-label graph cut energy optimization framework and a roof structure graph(RSG)model is then constructed to describe the roof topological geometry with common adjacency,symmetry,and convexity rules.Moreover,a progressive roof decomposition and refinement are performed,generating a hierarchical representation of the 3D roof structure models.Finally,a visual plane fitted residual or area constraint process is adopted to generate the RSG model with different levels of details.Results Two airborne laser scanning datasets with different point densities and roof styles were tested,and the performance evaluation metrics were obtained by International Society for Photogrammetry and Remote Sensing,achieving a correctness and accuracy of 97.7%and 0.29m,respectively.Conclusions The standardized assessment results demonstrate the effectiveness and robustness of the proposed approach,showing its ability to generate a variety of structural models,even with missing data.

3D打印应用于膝关节骨缺损修复重建1例

目的报道3D打印模块辅助创伤性骨缺损修复重建术的方法和疗效。方法纳入1名年轻女性病例,因交通事故致左膝开放性骨折并左股骨外髁缺损。Ⅰ期清创术后创面闭合2个月,基于患者CT原始数据,三维重建出股骨及股骨关节面的曲面模型,快速成型技术打印出相应的模块,术中辅助同种异体骨塑形,实现骨结构的修复重建。术后长期随访,通过主观感觉、影像表现、HSS评分等评价该治疗方法的有效性。结果术后1个月至8年随访期间,患肢疼痛、功能、关节活动度、肌力、膝关节屈曲畸形、膝关节不稳定等均呈良性发展,患肢活动功能满足日常需要。部分植骨块吸收,未见其它致残因素及明显不良事件发生。术后1月、2年、8年,HSS评分分别为44分、58分、83分。结论3D打印模块辅助手术能够提供个性化治疗,为创伤性骨缺损修复重建手术提高准确性,疗效肯定。

Modeling of a Linear Scanning 3D Vision Coordinate Measurement System

This paper theoretically analyzes and researches the coordinate frames of a 3D vision scanning system, establishes the mathematic model of a system scanning process, derives the relationship between the general non-orthonormal sensor coordinate system and the machine coordinate system and the coordinate transformation matrix of the extrinsic calibration for the system.