Multi-View Image-Based 3D Reconstruction in Indoor Scenes:A Survey

Three-dimensional reconstruction technology plays an important role in indoor scenes by converting objects and structures in indoor environments into accurate 3D models using multi-view RGB images.It offers a wide range of applications in fields such as virtual reality,augmented reality,indoor navigation,and game development.Existing methods based on multi-view RGB images have made significant progress in 3D reconstruction.These image-based reconstruction methods not only possess good expressive power and generalization performance,but also handle complex geometric shapes and textures effectively.Despite facing challenges such as lighting variations,occlusion,and texture loss in indoor scenes,these challenges can be effectively addressed through deep neural networks,neural implicit surface representations,and other techniques.The technology of indoor 3D reconstruction based on multi-view RGB images has a promising future.It not only provides immersive and interactive virtual experiences but also brings convenience and innovation to indoor navigation,interior design,and virtual tours.As the technology evolves,these image-based reconstruction methods will be further improved to provide higher quality and more accurate solutions to indoor scene reconstruction.

Development of vehicle-recognition method on water surfaces using LiDAR data:SPD^(2)(spherically stratified point projection with diameter and distance)

Swarm robot systems are an important application of autonomous unmanned surface vehicles on water surfaces.For monitoring natural environments and conducting security activities within a certain range using a surface vehicle,the swarm robot system is more efficient than the operation of a single object as the former can reduce cost and save time.It is necessary to detect adjacent surface obstacles robustly to operate a cluster of unmanned surface vehicles.For this purpose,a LiDAR(light detection and ranging)sensor is used as it can simultaneously obtain 3D information for all directions,relatively robustly and accurately,irrespective of the surrounding environmental conditions.Although the GPS(global-positioning-system)error range exists,obtaining measurements of the surface-vessel position can still ensure stability during platoon maneuvering.In this study,a three-layer convolutional neural network is applied to classify types of surface vehicles.The aim of this approach is to redefine the sparse 3D point cloud data as 2D image data with a connotative meaning and subsequently utilize this transformed data for object classification purposes.Hence,we have proposed a descriptor that converts the 3D point cloud data into 2D image data.To use this descriptor effectively,it is necessary to perform a clustering operation that separates the point clouds for each object.We developed voxel-based clustering for the point cloud clustering.Furthermore,using the descriptor,3D point cloud data can be converted into a 2D feature image,and the converted 2D image is provided as an input value to the network.We intend to verify the validity of the proposed 3D point cloud feature descriptor by using experimental data in the simulator.Furthermore,we explore the feasibility of real-time object classification within this framework.

First-principles Study on Geometric and Electronic Structures of Si（111）-√7× √3-In Surface Reconstruction

In order to determine the structures of Si（111）-√7 √3-In surfaces and to understand their electronic properties, we construct six models of both hexagonal and rectangular types and perform first-principles calculations. Their scanning tunneling microscopic images and work functions are simulated and compared with experimental results. In this way, the hex-H3＇ and rect-T1 models are identified as the experimental configurations for the hexagonal and rectangular types, respectively. The structural evolution mechanism of the In/Si（lll） surface with indium coverage around 1.0 monolayer is discussed. The 4×1 and -√7× √3 phases are suggested to have two different types of evolution mechanisms, consistent with experimental results.

3D surface reconstruction based on binocular vision using structured light

A 3D surface reconstruction method using a binocular stereo vision technology and a coded structured light,which combines a gray code with phase-shift has been studied.The accuracy of the 3 D surface reconstruction mainly depends on the decoding of gray code views and phase-shift views.In order to find the boundary accurately,gray code patterns and their inverses are projected onto a human eye plaster model.The period dislocation between the gray code views and the phase-shift views in the course of decoding has been analyzed and a new method has been proposed to solve it.The splicing method is based on feature points.The result of the 3D surface reconstruction shows the accuracy and reliability of our method.

3D reconstruction of complex curved objects from line drawings

An active research topic in computer vision and graphics is developing algorithms that can reconstruct the 3D surface of curved objects from line drawings. There are a number of algorithms have been dedicated to solve this problem, but they can＇t solve this problem when the geometric structure of a curved object becomes complex. This paper proposes a novel approach to reconstructing a complex curved 3D object from single 2D line drawings. Our approach has three steps： （1） decomposing a complex line drawing into several simpler line drawings and transforming them into polyhedron; （2） reconstructing the 3D wireframe of curved object from these simpler line drawings and generating the curved faces; （3） combining the 3D objects into the complete objects. A number of examples are given to demonstrate the ability of our approach to successfully perform reconstruction of curved objects which are more complex than previous methods.

New Virtual Cutting Algorithms for 3D Surface Model Reconstructed from Medical Images

This paper proposes a practical algorithms of plane cutting, stereo clipping and arbitrary cutting for 3D surface model reconstructed from medical images. In plane cutting and stereo clipping algorithms, the 3D model is cut by plane or polyhedron. Lists of edge and vertex in every cut plane are established. From these lists the boundary contours are created and their relationship of embrace is ascertained. The region closed by the contours is triangulated using Delaunay triangulation algorithm. Arbitrary cutting operation creates cutting curve interactively. The cut model still maintains its correct topology structure. With these operations, tissues inside can be observed easily and it can aid doctors to diagnose. The methods can also be used in surgery planning of radiotherapy.

Fast Mesh Reconstruction from Single View Based on GCN and Topology Modification

3D reconstruction based on single view aims to reconstruct the entire 3D shape of an object from one perspective.When existing methods reconstruct the mesh surface of complex objects,the surface details are difficult to predict and the reconstruction visual effect is poor because the mesh representation is not easily integrated into the deep learning framework;the 3D topology is easily limited by predefined templates and inflexible,and unnecessary mesh self-intersections and connections will be generated when reconstructing complex topology,thus destroying the surface details;the training of the reconstruction network is limited by the large amount of information attached to the mesh vertices,and the training time of the reconstructed network is too long.In this paper,we propose a method for fast mesh reconstruction from single view based on Graph Convolutional Network(GCN)and topology modification.We use GCN to ensure the generation of high-quality mesh surfaces and use topology modification to improve the flexibility of the topology.Meanwhile,a feature fusion method is proposed to make full use of the features of each stage of the image hierarchically.We use 3D open dataset ShapeNet to train our network and add a new weight parameter to speed up the training process.Extensive experiments demonstrate that our method can not only reconstruct object meshes on complex topological surfaces,but also has better qualitative and quantitative results.

Algorithm and System of Scanning Color 3D Objects

This paper presents a complete system for scanning the geometry and texture of a large 3D object, then the automatic registration is performed to obtain a whole realistic 3D model. This system is composed of one line strip laser and one color CCD camera. The scanned object is pictured twice by a color CCD camera. First, the texture of the scanned object is taken by a color CCD camera. Then the 3D information of the scanned object is obtained from laser plane equations. This paper presents a practical way to implement the three dimensional measuring method and the automatic registration of a large 3D object and a pretty good result is obtained after experiment verification.

一种医学图像的3-D表面重建方法

等值面绘制算法是医学图像表面重建的一种重要方法,但是它需要用户根据经验设定阈值。本文运用OTSU算法来获取体数据各类组织的灰度阈值,由此求得各类组织的灰度均值,并以此作为等值面阈值。在VTK中的重建实例表明,该方法能够体现出不同组织影像的灰度分布特点,使重建后的图像信息更丰富、表面更平滑。

Topology optimization of 3D structures with design-dependent loads

Topology optimization of continuum structures with design-dependent loads has long been a challenge. In this paper, the topology optimization of 3D structures subjected to design-dependent loads is investigated. A boundary search scheme is proposed for 3D problems, by means of which the load surface can be identified effectively and efficiently, and the difficulties arising in other approaches can be overcome. The load surfaces are made up of the boundaries of finite elements and the loads can be directly applied to corresponding element nodes, which leads to great convenience in the application of this method. Finally, the effectiveness and efficiency of the proposed method is validated by several numerical examples.

Exploring Local Regularities for 3D Object Recognition

In order to find better simplicity measurements for 3D object recognition, a new set of local regularities is developed and tested in a stepwise 3D reconstruction method, including localized minimizing standard deviation of angles（L-MSDA）, localized minimizing standard deviation of segment magnitudes（L-MSDSM）, localized minimum standard deviation of areas of child faces （L-MSDAF）, localized minimum sum of segment magnitudes of common edges （L-MSSM）, and localized minimum sum of areas of child face （L-MSAF）. Based on their effectiveness measurements in terms of form and size distortions, it is found that when two local regularities： L-MSDA and L-MSDSM are combined together, they can produce better performance. In addition, the best weightings for them to work together are identified as 10% for L-MSDSM and 90% for L-MSDA. The test results show that the combined usage of L-MSDA and L-MSDSM with identified weightings has a potential to be applied in other optimization based 3D recognition methods to improve their efficacy and robustness.

3-D Visualization of Medical Images with Arbitrary Sections

In this paper, with the general retrospect to the research on surface reconstruction and the marching cubes algorithm, we gave detailed description of an algorithm on the construction of object surfaces. The possible ambiguity problem in the original marching cubes algorithm was eliminated by its index mechanism. Some results on the MRI images were presented. Based on extracting and clipping contours from a set of medial slice images and setting the patch vertices values according to the gray images, this algorithm may be applied to form the arbitrary section images with three dimensional effects. It can also enhance the visual effect and interpretation of medical data.

NUMERICAL SIMULATION OF 3-D TURBULENT FLOWS OVER DREDGED TRENCHES

A 3- D free surface flow in open channels based on the Reynolds equations with the k-ε turbulence closure model is presented in this paper. Insted of the 'rigid lid' approximation, the solution of the free surface equation is implemented in the velocity-pressure iterative procedure on the basis of the conventional SIMPLE method. This model was used to compute the flow in rectangular channels with trenches dredged across the bottom. The velocity, eddy viscosity coefficient, turbulent shear stress, turbulent kinetic energy and elevation of the free surface can be obtained. The computed results are in good agreement with previous experimental data.

基于Shape-from-Shading的月球表面三维形状恢复算法研究

提出了一种基于Shape-from-Shading的月球表面三维形状恢复算法。首先分析了在太阳光照射下月球表面成像模型,建立了使用Lommel-Seeliger反射模型描述的反射图方程。然后用有限差分近似微分运算,将反射图方程所示的一阶变系数线性偏微分方程进行离散化处理,得到关于表面高度函数的代数方程。进而采用超松弛迭代法进行求解,获得月球表面三维高度函数值。最后使用合成图像和实际月球图像进行三维形状恢复仿真实验。实验结果表明提出的算法可以有效地恢复月球表面三维形状。

Scattering simulation and reconstruction of a 3D complex target above an underlying surface using SIMO radar with plane array

A SIMO(single input and multiple output) system of a step-frequency(SF) radar is used.It works in downward-looking spotlight mode and moves within a 2D synthetic plane array.A 3D(three-dimensional) matrix of bistatic scattering fields is produced in both the amplitude and phase from a 3D complex-shaped electric-large target above background surface.In numerical simulation,the bidirectional analytic ray tracing(BART) method is applied to calculate bistatic scattering in the SIMO observations from a volumetric target above background rough surface.An improved 3D RMA(range migration algorithm) is then utilized to make the imaging.Its 3D imaging is applied to reconstruct the target profile.As validation and comparison,the scattering fields of some simple targets are computed with comparisons of the BART and FEKO software.The SIMO techniques of imaging and reconstruction for a 3D target,such as a tank-like model over rough surface,are presented.

基于结构光和深度神经网络的3维面形重建

为了提高基于结构光法的3维重建精度,采用机器学习中的回归模型对物体进行了3维形貌测量,通过以单目式获取对象高度点不同方向的光强信息簇样本,将其作为回归模型的训练集,在训练好回归模型后,直接建立起条纹图案的光强信息分布与对象高度之间的映射函数关系,完成对目标的3维测量;将调制条纹光数值信息以特征形式导入回归模型,获得端到端高度信息,验证了机器学习的神经网络回归模型在3维面形重建上的可行性。结果表明,该模型即使在投影特征模糊或噪音较大的情况也能较精确地重建3维面形,平均重建误差为1.40×10^(-4)mm,优于一般面形重建方法的数据。该研究为物体在强干扰条件下的单目式高精度3维面形重建提供了参考,简化了繁琐的计算过程和测量过程,提高了测量精度。

High-quality indoor scene 3D reconstruction with RGB-D cameras:A brief review

High-quality 3D reconstruction is an important topic in computer graphics and computer vision with many applications,such as robotics and augmented reality.The advent of consumer RGB-D cameras has made a profound advance in indoor scene reconstruction.For the past few years,researchers have spent significant effort to develop algorithms to capture 3D models with RGB-D cameras.As depth images produced by consumer RGB-D cameras are noisy and incomplete when surfaces are shiny,bright,transparent,or far from the camera,obtaining highquality 3D scene models is still a challenge for existing systems.We here review high-quality 3D indoor scene reconstruction methods using consumer RGB-D cameras.In this paper,we make comparisons and analyses from the following aspects:(i)depth processing methods in 3D reconstruction are reviewed in terms of enhancement and completion,(ii)ICP-based,feature-based,and hybrid methods of camera pose estimation methods are reviewed,and(iii)surface reconstruction methods are reviewed in terms of surface fusion,optimization,and completion.The performance of state-of-the-art methods is also compared and analyzed.This survey will be useful for researchers who want to follow best practices in designing new high-quality 3D reconstruction methods.

Automatic object annotation in streamed and remotely explored large 3D reconstructions

We introduce a novel framework for 3 D scene reconstruction with simultaneous object annotation,using a pre-trained 2 D convolutional neural network(CNN),incremental data streaming,and remote exploration,with a virtual reality setup.It enables versatile integration of any 2 D box detection or segmentation network.We integrate new approaches to(i)asynchronously perform dense 3 D-reconstruction and object annotation at interactive frame rates,(ii)efficiently optimize CNN results in terms of object prediction and spatial accuracy,and(iii)generate computationally-efficient colliders in large triangulated3 D-reconstructions at run-time for 3 D scene interaction.Our method is novel in combining CNNs with long and varying inference time with live 3 D-reconstruction from RGB-D camera input.We further propose a lightweight data structure to store the 3 D-reconstruction data and object annotations to enable fast incremental data transmission for real-time exploration with a remote client,which has not been presented before.Our framework achieves update rates of 22 fps(SSD Mobile Net)and 19 fps(Mask RCNN)for indoor environments up to 800 m^(3).We evaluated the accuracy of 3 D-object detection.Our work provides a versatile foundation for semantic scene understanding of large streamed3 D-reconstructions,while being independent from the CNN’s processing time.Source code is available for non-commercial use.

一种基于三维Delaunay三角化的曲面重建算法

提出一种基于三维Delaunay三角化的区域增长式曲面重建方法。该方法以空间点云的Delaunay三角化为基础,结合局部区域增长的曲面构造,较以往方法具有人为参与更少、适用范围更广的优点。算法采用增量式插入点的方式构建空间Delaunay划分,采用广度优先算法,以外接圆最小为准则从Delaunay三角化得到的四面体中抽取出合适的三角片构成曲面。该算法的设计无须计算原始点集的法矢,且孔洞系数对重建的结果影响很小,重建出的三角网格面更符合原始曲面的几何特征。无论待建曲面是否是封闭曲面,本算法均可获得较好的重建效果。

基于IDL的面绘制三维重构算法研究及应用

IDL中PolyPaint面绘制算法具有多边形拓扑一致性、顶点查找边索引及顶点数目最大化等特点,可避免经典MarchingCubes可能出现的空洞问题,并提高绘制效率。提出了医学CT图像面绘制三维重构的实现方法,并与VTK的绘制时间和效果进行对比,实验结果表明IDL在面绘制时间上有较大优势。