3D reconstruction method based on contour features

To guarantee the accuracy and real-time of the 3D reconstruction method for outdoor scene,an algorithm based on region segmentation and matching was proposed.Firstly,on the basis of morphological gradient information,obtained by comparing color weight gradient images and proposing a multi-threshold segmentation,scene contour features were extracted by a watershed algorithm and a fuzzy c-means clustering algorithm.Secondly,to reduce the search area,increase the correct matching ratio and accelerate the matching speed,the region constraint was established according to a region＇s local position,area and gray characteristics,the edge pixel constraint was established according to the epipolar constraint and the continuity constraint.Finally,by using the stereo matching edge pixel pairs,their 3D coordinates were estimated according to the binocular stereo vision imaging model.Experimental results show that the proposed method can yield a high stereo matching ratio and reconstruct a 3D scene quickly and efficiently.

A fast, accurate and dense feature matching algorithm for aerial images

Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.

Illumination-robust and Anti-blur Feature Descriptors for Image Matching in Abdomen Reconstruction

This paper puts forward a method for abdomen panorama reconstruction based on a stereo vision system. For the purpose of recovering the abdomen completely and accurately under the condition of actual photographing with illumination variance and blur noise, some innovative combined feature descriptors are presented on the basis of Hu-moment invariants. Furthermore, considering the study on the abdomen surface reconstruction, a circle template which is divided into 6 sectors is designed. It is noted that a descriptor merely using gray intensity is not able to provide sufficient information for feature description. Consequently, the sector entropy which denotes the structure characteristics is drawn into the feature descriptor. By means of the combined effect of the gray intensity and the sector entropy, the similarity measurement is conducted for the final abdomen reconstruction. The experimental results reveal that the proposed method can acquire a high precision of abdomen reconstruction similar to the 3D scanner. This stereo vision system has wide practicability in the field of clothing.

A new navigation approach of terrain contour matching based on 3-D terrain reconstruction from onboard image sequence

This article presents a passive navigation method of terrain contour matching by reconstructing the 3-D terrain from the image sequence(acquired by the onboard camera).To achieve automation and simultaneity of the image sequence processing for navigation,a correspondence registration method based on control points tracking is proposed which tracks the sparse control points through the whole image sequence and uses them as correspondence in the relation geometry solution.Besides,a key frame selection method based on the images overlapping ratio and intersecting angles is explored,thereafter the requirement for the camera system configuration is provided.The proposed method also includes an optimal local homography estimating algorithm according to the control points,which helps correctly predict points to be matched and their speed corresponding.Consequently,the real-time 3-D terrain of the trajectory thus reconstructed is matched with the referenced terrain map,and the result of which provides navigating information.The digital simulation experiment and the real image based experiment have verified the proposed method.

Vision navigation for aircrafts based on 3D reconstruction from real-time image sequences

In this paper,we propose a novel vision navigation method based on three-dimensional(3D)reconstruction from real-time image sequences.It adapts 3D reconstruction and terrain matching to establish the correspondence between image points and3D space points and the terrain reference(by using a digital elevation map(DEM)).An adaptive weighted orthogonal iterative pose estimation method is employed to calculate the position and attitude angle of the aircraft.Synthesized and real experiments show that the proposed method is capable of providing accurate navigation parameters for a long-endurance flight without using a global positioning system or an inertial navigation system(INS).Moreover,it can be combined with an INS to achieve an improved navigation result.

Feature-based RGB-D camera pose optimization for real-time 3D reconstruction

In this paper we present a novel featurebased RGB-D camera pose optimization algorithm for real-time 3D reconstruction systems. During camera pose estimation, current methods in online systems suffer from fast-scanned RGB-D data, or generate inaccurate relative transformations between consecutive frames. Our approach improves current methods by utilizing matched features across all frames and is robust for RGB-D data with large shifts in consecutive frames. We directly estimate camera pose for each frame by efficiently solving a quadratic minimization problem to maximize the consistency of3 D points in global space across frames corresponding to matched feature points. We have implemented our method within two state-of-the-art online 3D reconstruction platforms. Experimental results testify that our method is efficient and reliable in estimating camera poses for RGB-D data with large shifts.

GeoGlue: feature matching with self-supervised geometric priors for high-resolution UAV images

We present GeoGlue,a novel method using high-resolution UAV imagery for accurate feature matching,which is normally challenging due to the complicated scenes.Current feature detection methods are performed without guidance of geometric priors(e.g.,geometric lines),lacking enough attention given to salient geometric features which are indispensable for accurate matching due to their stable existence across views.In this work,geometric lines arefirstly detected by a CNN-based geometry detector(GD)which is pre-trained in a self-supervised manner through automatically generated images.Then,geometric lines are naturally vectorized based on GD and thus non-significant features can be disregarded as judged by their disordered geometric morphology.A graph attention network(GAT)is utilized forfinal feature matching,spanning across the image pair with geometric priors informed by GD.Comprehensive experiments show that GeoGlue outperforms other state-of-the-art methods in feature-matching accuracy and performance stability,achieving pose estimation with maximum rotation and translation errors under 1%in challenging scenes from benchmark datasets,Tanks&Temples and ETH3D.This study also proposes thefirst self-supervised deep-learning model for curved line detection,generating geometric priors for matching so that more attention is put on prominent features and improving the visual effect of 3D reconstruction.

A method of 3D reconstruction based on panoramic cameras

In this paper,a method of 3D reconstruction from two images acquired by two panoramic cameras is presented.Firstly,the features of the reconstruction object detected in each image are matched through the DP matching method.Secondly,optical correction is carried out on two cameras,and the internal parameters of panoramic cameras can be calculated.Finally,according to the calibration method,the geometric relationship between corresponding points in space and in two panoramic images is deduced.The results indicate that the method of 3D reconstruction based on two panoramic cameras is simple,and the accuracy can reach 98.82%.

Improvement of Binocular Reconstruction Algorithm for Measuring 3D Pavement Texture Using a Single Laser Line Scanning Constraint

The dense and accurate measurement of 3D texture is helpful in evaluating the pavement function.To form dense mandatory constraints and improve matching accuracy,the traditional binocular reconstruction technology was improved threefold.First,a single moving laser line was introduced to carry out global scanning constraints on the target,which would well overcome the difficulty of installing and recognizing excessive laser lines.Second,four kinds of improved algorithms,namely,disparity replacement,superposition synthesis,subregion segmentation,and subregion segmentation centroid enhancement,were established based on different constraint mechanism.Last,the improved binocular reconstruction test device was developed to realize the dual functions of 3D texture measurement and precision self-evaluation.Results show that compared with traditional algorithms,the introduction of a single laser line scanning constraint is helpful in improving the measurement’s accuracy.Among various improved algorithms,the improvement effect of the subregion segmentation centroid enhancement method is the best.It has a good effect on both overall measurement and single pointmeasurement,which can be considered to be used in pavement function evaluation.

3D Echocardiogram Reconstruction Employing a Flip Directional Texture Pyramid

Three dimensional(3D)echocardiogram enables cardiologists to visua-lize suspicious cardiac structures in detail.In recent years,this three-dimensional echocardiogram carries important clinical value in virtual surgical simulation.However,this 3D echocardiogram involves a trade-off difficulty between accu-racy and efficient computation in clinical diagnosis.This paper presents a novel Flip Directional 3D Volume Reconstruction(FD-3DVR)method for the recon-struction of echocardiogram images.The proposed method consists of two main steps:multiplanar volumetric imaging and 3D volume reconstruction.In the crea-tion of multiplanar volumetric imaging,two-dimensional(2D)image pixels are mapped into voxels of the volumetric grid.As the obtained slices are discontin-uous,there are some missing voxels in the volume data.To restore the structural and textural information of 3D ultrasound volume,the proposed method creates a volume pyramid in parallel with theflip directional texture pyramid.Initially,the nearest neighbors of missing voxels in the multiplanar volumetric imaging are identified by 3D ANN(Approximate Nearest Neighbor)patch matching method.Furthermore,aflip directional texture pyramid is proposed and aggregated with distance in patch matching tofind out the most similar neighbors.In the recon-struction step,structural and textural information obtained from differentflip angle directions can reconstruct 3D volume well with the desired accuracy.Com-pared with existing 3D reconstruction methods,the proposed Flip Directional 3D Volume Reconstruction(FD-3DVR)method provides superior performance for the mean peak signal-to-noise ratio(40.538 for the proposed method I and 39.626 for the proposed method II).Experimental results performed on the cardi-ac datasets demonstrate the efficiency of the proposed method for the reconstruc-tion of echocardiogram images.

Wheel center detection based on stereo vision

As the location of the wheel center is the key to accurately measuring the wheelbase, the wheelbase difference and the wheel static radius, a high-precision wheel center detection method based on stereo vision is proposed. First, according to the prior information, the contour of the wheel hub is extracted and fitted as an ellipse curve, and the ellipse fitting equation can be obtained. Then, a new un-tangent constraint is adopted to improve the ellipse matching precision. Finally, the 3D coordinates of the wheel center can be reconstructed by the spatial circle projection algorithm with low time complexity and high measurement accuracy. Simulation experiments verify that compared with the ellipse center reconstruction algorithm and the planar constraint optimization algorithm, the proposed method can acquire the 3D coordinates of the spatial circle more exactly. Furthermore, the measurements of the wheelbase, the wheelbase difference and the wheel static radius for three types of vehicles demonstrate the effectiveness of the proposed method for wheel center detection.

Using MC Algorithm to Implement 3D Image Reconstruction for Yunnan Weather Radar Data

3D image reconstruction for weather radar data can not only help the weatherman to improve the forecast efficiency and accuracy, but also help people to understand the weather conditions easily and quickly. Marching Cubes (MC) algorithm in the surface rendering has more excellent applicability in 3D reconstruction for the slice images;it may shorten the time to find and calculate the isosurface from raw volume data, reflect the shape structure more accurately. In this paper, we discuss a method to reconstruct the 3D weather cloud image by using the proposed Cube Weighting Interpolation (CWI) and MC algorithm. Firstly, we detail the steps of CWI, apply it to project the raw radar data into the cubes and obtain the equally spaced cloud slice images, then employ MC algorithm to draw the isosurface. Some experiments show that our method has a good effect and simple operation, which may provide an intuitive and effective reference for realizing the 3D surface reconstruction and meteorological image stereo visualization.

Scene Visual Perception and AR Navigation Applications

With the rapid popularization of mobile devices and the wide application of various sensors,scene perception methods applied to mobile devices occupy an important position in location-based services such as navigation and augmented reality(AR).The development of deep learning technologies has greatly improved the visual perception ability of machines to scenes.The basic framework of scene visual perception,related technologies and the specific process applied to AR navigation are introduced,and future technology development is proposed.An application(APP)is designed to improve the application effect of AR navigation.The APP includes three modules:navigation map generation,cloud navigation algorithm,and client design.The navigation map generation tool works offline.The cloud saves the navigation map and provides navigation algorithms for the terminal.The terminal realizes local real-time positioning and AR path rendering.

一种新的基于R-D分析的重力匹配辅助导航算法

针对现有重力辅助导航匹配算法匹配精度、匹配率受惯导初始位置误差影响较大以及实时性较差等不足,提出一种实时重力辅助导航匹配算法.新算法利用惯导输出的相对位置信息,提取半径、方向角约束模型,然后通过对相似性进行度量,筛选出待匹配点,得到最优匹配参数,该算法可以在惯导初始误差较大的情况下进行实时、高效匹配.实验结果表明,新算法对惯导初始误差不敏感、鲁棒性好,而且精度高、实时性强.

基于二维DR的颅面硬组织与软组织拟合的研究

目的 :以数字化X线摄影片(digital radiograph,DR)图像和激光扫描为数据源,三维重建颅面硬组织和软组织,并进行硬软组织的有效拟合,以期为治疗方案确定、手术方法选择和提高医患交流的水平提供平台。方法:在志愿者面部贴定铅点,拍摄头颅正侧位DR片。利用基于二维DR数据的颅面硬组织三维重建系统进行硬组织形变建模,获得颅面部硬组织模型。利用激光扫描数据进行面部软组织三维重建,获得面部软组织三维模型。然后通过铅点坐标的匹配,将颅面硬组织与面部软组织进行拟合。结果:分别重建了能反映真实颅面部组织结构的硬组织三维模型和面部软组织三维模型,并实现了颅面部硬组织与软组织的有效拟合。结论:本研究实现了颅颌面三维结构的有效重建和拟合,为进一步进行正颌手术模拟和容貌预测奠定了基础。拟合结果真实可靠,能够用于临床。

A Skeletal Camera Network for Close-range Images with a Data Driven Approach in Analyzing Stereo Configuration

Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,the base-height ratio,intersection angle,overlap,and ground control points,etc.,which are rarely quantified in real-world applications.To answer this question,in this paper,we take a data-driven approach by analyzing hundreds of terrestrial stereo image configurations through a typical SfM algorithm.Two main meta-parameters with respect to base-height ratio and intersection angle are analyzed.Following the results,we propose a Skeletal Camera Network(SCN)and embed it into the SfM to lead to a novel SfM scheme called SCN-SfM,which limits tie-point matching to the remaining connected image pairs in SCN.The proposed method was applied in three terrestrial datasets.Experimental results have demonstrated the effectiveness of the proposed SCN-SfM to achieve 3D geometry with higher accuracy and fast time efficiency compared to the typical SfM method,whereas the completeness of the geometry is comparable.

基于最优几何匹配的时间连贯3D动画重建

为了优化RGB-D数据的3D重建,提出一种时间连贯的3D重建方法。在数据采集后,从映射到深度数据的RGB数据中提取光学特征点,以得到两个帧之间的初始稀疏3D对应关系。利用最优几何匹配程序进行特征点细化,使用这些特征点对连续帧中的两个3D点云进行独立于帧分辨率的匹配;利用运动向量对齐方法重建时间连贯的3D动画。实验结果表明,所提方法可以重建时间连贯的3D动画。与其他方法相比,所提方法数据丢失较少,在平均误差方面具有一定优势,而且在计算方面也具有高效性。

基于正交匹配追踪算法的虚拟内插空间平滑DOA估计

针对正交匹配追踪(Orthogonal Matching Pursuit, OMP)算法在实际的波达方向(Direction of Arrival, DOA)估计应用中,测向分辨率较低且在相干信号环境下估计性能较差的问题,提出了一种虚拟阵列空间平滑DOA估计算法.以互质阵列为模型,通过在正交匹配追踪算法重构信号的基础上,使用虚拟内插的方法,构造均匀虚拟阵列,能够提高波达方向的估计精度,加入空间平滑算法,可以分辨相干信号.仿真结果表明,使用虚拟内插算法之后,空间谱图波峰更尖锐,且在信噪比为0 dB时,最高探测精度提高到2°,测向分辨率明显有所提升,同时加入空间平滑算法,可以有效地分辨相干信号,且在信噪比为0 dB时,最高探测精度提高为4.7°,验证了本文方法的有效性.

最优几何匹配程序下重建时间连贯3D动画的研究

对RGB-D数据的三维重建进行优化,由此提出基于时间连贯的重建策略。在完成数据收集之后,对其中的RGB数据进行光学特征点的提取,由此获取两帧相互之间的初始系数3D关系。在对这些特征点进行细化时,则运用了最优几何匹配系统,然后利用特征点对连续帧的三维点云展开匹配,这个过程与帧分辨率独立;最后完成时间连贯的3D动画重建,具体是借助于运动向量对齐策略。通过实验得出,本文提出的策略能够对该3D动画进行重建。相较于其他方法,数据利用率更高,同时还有较高的精准度,在计算效率方面也具有一定优势。

基于细化等值面拓扑的病灶切片三维重建方法

脑组织病灶切片的三维重构对于了解神经胶质神经瘤状态具有重要意义,可用于鉴别诊断、手术模拟等。移动立方体(MC)算法是经典的多边形曲面重建算法,具有简单易实现的优点,但其存在低效且梯级现象明显的情况。针对该问题,本研究提出一种细化等值面拓扑的病灶切片空间堆叠重建方法(SSR-RI),旨在实现拓扑构形与运行效率的优化。SSR-RI通过构建空间坐标系对磁共振图像(MRI)切片邻近图进行处理,为了改善双线性插值法中图像分量易受损的问题,提出一种自适应空间插值法,根据灰度值变化自适应选择插值点,对周围进行扩充。在结合等法线顶点的基础上,设计了一种细化等值面提取方式的堆叠重建方法,以提高堆叠速度,并减少梯级问题。为了进一步优化SSR-RI的三维重建效果,提出一种改进的局部反射光照法(PR)以绘制三维病变,利用镜面颜色反射(SCR)与镜面指数(SE)对重建体进行渲染优化。使用公开脑肿瘤分割数据集BraTS的618例病例开展三维重建实验,以验证所提出方法性能。实验结果显示,所提算法的重建时间只需2.124 s,F-score值达到0.845,SSIM值达到0.81,相较于MC算法减少了38%的重建时间,F-score值和SSIM值分别提高了30.89%和38.4%。重构体序列间结构紧密,视觉效果更富立体感和纹理感,有效地提高了三维重建的绘制效率。