The concrete aging problem has gained more attention in recent years as more bridges and tunnels in the United States lack proper maintenance. Though the Federal Highway Administration requires these public concrete s...The concrete aging problem has gained more attention in recent years as more bridges and tunnels in the United States lack proper maintenance. Though the Federal Highway Administration requires these public concrete structures to be inspected regularly, on-site manual inspection by human operators is time-consuming and labor-intensive. Conventional inspection approaches for concrete inspection, using RGB imagebased thresholding methods, are not able to determine metric information as well as accurate location information for assessed defects for conditions. To address this challenge, we propose a deep neural network(DNN) based concrete inspection system using a quadrotor flying robot(referred to as City Flyer) mounted with an RGB-D camera. The inspection system introduces several novel modules. Firstly, a visual-inertial fusion approach is introduced to perform camera and robot positioning and structure 3 D metric reconstruction. The reconstructed map is used to retrieve the location and metric information of the defects.Secondly, we introduce a DNN model, namely Ada Net, to detect concrete spalling and cracking, with the capability of maintaining robustness under various distances between the camera and concrete surface. In order to train the model, we craft a new dataset, i.e., the concrete structure spalling and cracking(CSSC)dataset, which is released publicly to the research community.Finally, we introduce a 3 D semantic mapping method using the annotated framework to reconstruct the concrete structure for visualization. We performed comparative studies and demonstrated that our Ada Net can achieve 8.41% higher detection accuracy than Res Nets and VGGs. Moreover, we conducted five field tests, of which three are manual hand-held tests and two are drone-based field tests. These results indicate that our system is capable of performing metric field inspection,and can serve as an effective tool for civil engineers.展开更多
The material flow in friction stir welded 2014 Al alloy has been investigated using a marker insert technique （MIT）. Results of the flow visualization show that the material flow is asymmetrical during the friction ...The material flow in friction stir welded 2014 Al alloy has been investigated using a marker insert technique （MIT）. Results of the flow visualization show that the material flow is asymmetrical during the friction stir welding （FSW） process and there are also significant differences in the flow patterns observed on advancing side and retreating side. On advancing side, some material transport forward and some move backward, but on retreating side, material only transport backward. At the top surface of the weld, significant material transport forward due to the action of the rotating tool shoulder. Combining the data from all the markers, a three-dituensional flow visualization, similar to the 3D image reconstruction technique, was obtained. The three-dimensional plot gives the tendency chart of material flow in friction stir welding process and from the plot it can be seen that there is a vertical, circular motion around the longitudinal axis of the weld. On the advancing side of the weld, the material is pushed downward but on the retreating side, the material is pushed toward the crown of the weld. The net result of the two relative motions in both side of the advancing and the retreating is that a circular motion comes into being. Comparatively, the material flow around the longitudinal axis is a secondary motion.展开更多
三维重建是计算机视觉研究的重要内容,在虚拟现实中应用广泛。2009年微软发布了3D体感摄影机Kinect,便于便宜快速地获取RGB与深度信息。介绍了空间坐标系旋转原理以及在Visual Studio 2010平台上实现三维人物的实时重建以及匹配颜色与...三维重建是计算机视觉研究的重要内容,在虚拟现实中应用广泛。2009年微软发布了3D体感摄影机Kinect,便于便宜快速地获取RGB与深度信息。介绍了空间坐标系旋转原理以及在Visual Studio 2010平台上实现三维人物的实时重建以及匹配颜色与深度信息的三维点云提取与雕刻。实验结果表明该方法能够有效地获取同颜色和三维信息,并且可以利用激光雕刻机打印出人物模型。展开更多
Recent monitoring techniques employ multiple sources of information for the characterization of the phenomenon to be studied, being the coupling and adjustment of multi-source data one of the first challenges to consi...Recent monitoring techniques employ multiple sources of information for the characterization of the phenomenon to be studied, being the coupling and adjustment of multi-source data one of the first challenges to consider and solve. The authors propose a new framework of the multi-source and multi-temporal data-oriented fusion for the characterization of landslide events. The main objective is to generate 3D virtual models(in the form of dense point clouds) and feed them back with the characteristic of soil and subsoil information. The scheme consists of three main steps. The first one is on-site data collection(geological characterization, geophysical measurements, GPS measurements, and UAV/drone mapping). The second step is generation of a high-resolution 3D virtual model(~1-inch spatial resolution) from the frames acquired through the UAV using the structure of motion(SfM) processing;the developed virtual model is optimized with GPS measurements to minimize geolocation error and eliminate distortions. The last step is assembling of the acquired data in the field and densified point cloud considering the different nature of the data, re-escalating procedure and the information stacking layer.展开更多
High-quality 3D reconstruction is an important topic in computer graphics and computer vision with many applications,such as robotics and augmented reality.The advent of consumer RGB-D cameras has made a profound adva...High-quality 3D reconstruction is an important topic in computer graphics and computer vision with many applications,such as robotics and augmented reality.The advent of consumer RGB-D cameras has made a profound advance in indoor scene reconstruction.For the past few years,researchers have spent significant effort to develop algorithms to capture 3D models with RGB-D cameras.As depth images produced by consumer RGB-D cameras are noisy and incomplete when surfaces are shiny,bright,transparent,or far from the camera,obtaining highquality 3D scene models is still a challenge for existing systems.We here review high-quality 3D indoor scene reconstruction methods using consumer RGB-D cameras.In this paper,we make comparisons and analyses from the following aspects:(i)depth processing methods in 3D reconstruction are reviewed in terms of enhancement and completion,(ii)ICP-based,feature-based,and hybrid methods of camera pose estimation methods are reviewed,and(iii)surface reconstruction methods are reviewed in terms of surface fusion,optimization,and completion.The performance of state-of-the-art methods is also compared and analyzed.This survey will be useful for researchers who want to follow best practices in designing new high-quality 3D reconstruction methods.展开更多
基金supported in part by the U.S.National Science Foundation(IIP-1915721)the U.S.Department of TransportationOffice of the Assistant Secretary for Research and Technology(USDOTOST-R)(69A3551747126)through INSPIRE University Transportation Center(http//inspire-utc.mst.edu)at Missouri University of Science and Technology。
文摘The concrete aging problem has gained more attention in recent years as more bridges and tunnels in the United States lack proper maintenance. Though the Federal Highway Administration requires these public concrete structures to be inspected regularly, on-site manual inspection by human operators is time-consuming and labor-intensive. Conventional inspection approaches for concrete inspection, using RGB imagebased thresholding methods, are not able to determine metric information as well as accurate location information for assessed defects for conditions. To address this challenge, we propose a deep neural network(DNN) based concrete inspection system using a quadrotor flying robot(referred to as City Flyer) mounted with an RGB-D camera. The inspection system introduces several novel modules. Firstly, a visual-inertial fusion approach is introduced to perform camera and robot positioning and structure 3 D metric reconstruction. The reconstructed map is used to retrieve the location and metric information of the defects.Secondly, we introduce a DNN model, namely Ada Net, to detect concrete spalling and cracking, with the capability of maintaining robustness under various distances between the camera and concrete surface. In order to train the model, we craft a new dataset, i.e., the concrete structure spalling and cracking(CSSC)dataset, which is released publicly to the research community.Finally, we introduce a 3 D semantic mapping method using the annotated framework to reconstruct the concrete structure for visualization. We performed comparative studies and demonstrated that our Ada Net can achieve 8.41% higher detection accuracy than Res Nets and VGGs. Moreover, we conducted five field tests, of which three are manual hand-held tests and two are drone-based field tests. These results indicate that our system is capable of performing metric field inspection,and can serve as an effective tool for civil engineers.
文摘The material flow in friction stir welded 2014 Al alloy has been investigated using a marker insert technique （MIT）. Results of the flow visualization show that the material flow is asymmetrical during the friction stir welding （FSW） process and there are also significant differences in the flow patterns observed on advancing side and retreating side. On advancing side, some material transport forward and some move backward, but on retreating side, material only transport backward. At the top surface of the weld, significant material transport forward due to the action of the rotating tool shoulder. Combining the data from all the markers, a three-dituensional flow visualization, similar to the 3D image reconstruction technique, was obtained. The three-dimensional plot gives the tendency chart of material flow in friction stir welding process and from the plot it can be seen that there is a vertical, circular motion around the longitudinal axis of the weld. On the advancing side of the weld, the material is pushed downward but on the retreating side, the material is pushed toward the crown of the weld. The net result of the two relative motions in both side of the advancing and the retreating is that a circular motion comes into being. Comparatively, the material flow around the longitudinal axis is a secondary motion.
文摘三维重建是计算机视觉研究的重要内容,在虚拟现实中应用广泛。2009年微软发布了3D体感摄影机Kinect,便于便宜快速地获取RGB与深度信息。介绍了空间坐标系旋转原理以及在Visual Studio 2010平台上实现三维人物的实时重建以及匹配颜色与深度信息的三维点云提取与雕刻。实验结果表明该方法能够有效地获取同颜色和三维信息,并且可以利用激光雕刻机打印出人物模型。
基金supported by the CONACYT Academic Fellowship(No.308896)
文摘Recent monitoring techniques employ multiple sources of information for the characterization of the phenomenon to be studied, being the coupling and adjustment of multi-source data one of the first challenges to consider and solve. The authors propose a new framework of the multi-source and multi-temporal data-oriented fusion for the characterization of landslide events. The main objective is to generate 3D virtual models(in the form of dense point clouds) and feed them back with the characteristic of soil and subsoil information. The scheme consists of three main steps. The first one is on-site data collection(geological characterization, geophysical measurements, GPS measurements, and UAV/drone mapping). The second step is generation of a high-resolution 3D virtual model(~1-inch spatial resolution) from the frames acquired through the UAV using the structure of motion(SfM) processing;the developed virtual model is optimized with GPS measurements to minimize geolocation error and eliminate distortions. The last step is assembling of the acquired data in the field and densified point cloud considering the different nature of the data, re-escalating procedure and the information stacking layer.
基金National Key R&D Program of China under Grant No.2018YFC2000600Open Projects Program of National Laboratory of Pattern Recognition under Grant No.202100009+1 种基金National Natural Science Foundation of China under Grant No.72071018Fundamental Research Funds for Central Universities under Grant No.2021TD006。
文摘High-quality 3D reconstruction is an important topic in computer graphics and computer vision with many applications,such as robotics and augmented reality.The advent of consumer RGB-D cameras has made a profound advance in indoor scene reconstruction.For the past few years,researchers have spent significant effort to develop algorithms to capture 3D models with RGB-D cameras.As depth images produced by consumer RGB-D cameras are noisy and incomplete when surfaces are shiny,bright,transparent,or far from the camera,obtaining highquality 3D scene models is still a challenge for existing systems.We here review high-quality 3D indoor scene reconstruction methods using consumer RGB-D cameras.In this paper,we make comparisons and analyses from the following aspects:(i)depth processing methods in 3D reconstruction are reviewed in terms of enhancement and completion,(ii)ICP-based,feature-based,and hybrid methods of camera pose estimation methods are reviewed,and(iii)surface reconstruction methods are reviewed in terms of surface fusion,optimization,and completion.The performance of state-of-the-art methods is also compared and analyzed.This survey will be useful for researchers who want to follow best practices in designing new high-quality 3D reconstruction methods.
