Optimized CUDA Implementation to Improve the Performance of Bundle Adjustment Algorithm on GPUs

The 3D reconstruction pipeline uses the Bundle Adjustment algorithm to refine the camera and point parameters. The Bundle Adjustment algorithm is a compute-intensive algorithm, and many researchers have improved its performance by implementing the algorithm on GPUs. In the previous research work, “Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm using GPUs,” the authors demonstrated first the Bundle Adjustment algorithmic performance improvement by reducing the mean square error using an additional radial distorting parameter and explicitly computed analytical derivatives and reducing the computational burden of the Bundle Adjustment algorithm using GPUs. The naïve implementation of the CUDA code, a speedup of 10× for the largest dataset of 13,678 cameras, 4,455,747 points, and 28,975,571 projections was achieved. In this paper, we present the optimization of the Bundle Adjustment algorithm CUDA code on GPUs to achieve higher speedup. We propose a new data memory layout for the parameters in the Bundle Adjustment algorithm, resulting in contiguous memory access. We demonstrate that it improves the memory throughput on the GPUs, thereby improving the overall performance. We also demonstrate an increase in the computational throughput of the algorithm by optimizing the CUDA kernels to utilize the GPU resources effectively. A comparative performance study of explicitly computing an algorithm parameter versus using the Jacobians instead is presented. In the previous work, the Bundle Adjustment algorithm failed to converge for certain datasets due to several block matrices of the cameras in the augmented normal equation, resulting in rank-deficient matrices. In this work, we identify the cameras that cause rank-deficient matrices and preprocess the datasets to ensure the convergence of the BA algorithm. Our optimized CUDA implementation achieves convergence of the Bundle Adjustment algorithm in around 22 seconds for the largest dataset compared to 654 seconds for the sequential implementation, resulting in a speedup of 30×. Our optimized CUDA implementation presented in this paper has achieved a 3× speedup for the largest dataset compared to the previous naïve CUDA implementation.

A Self-calibration Bundle Adjustment Algorithm Based on Block Matrix Cholesky Decomposition Technology

In this study,the problem of bundle adjustment was revisited,and a novel algorithm based on block matrix Cholesky decomposition was proposed to solve the thorny problem of self-calibration bundle adjustment.The innovation points are reflected in the following aspects:①The proposed algorithm is not dependent on the Schur complement,and the calculation process is simple and clear;②The complexities of time and space tend to O(n)in the context of world point number is far greater than that of images and cameras,so the calculation magnitude and memory consumption can be reduced significantly;③The proposed algorithm can carry out self-calibration bundle adjustment in single-camera,multi-camera,and variable-camera modes;④Some measures are employed to improve the optimization effects.Experimental tests showed that the proposed algorithm has the ability to achieve state-of-the-art performance in accuracy and robustness,and it has a strong adaptability as well,because the optimized results are accurate and robust even if the initial values have large deviations from the truth.This study could provide theoretical guidance and technical support for the image-based positioning and 3D reconstruction in the fields of photogrammetry,computer vision and robotics.

Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm Using GPUs

Bundle adjustment is a camera and point refinement technique in a 3D scene reconstruction pipeline. The camera parameters and the 3D points are refined by minimizing the difference between computed projection and observed projection of the image points formulated as a non-linear least-square problem. Levenberg-Marquardt method is used to solve the non-linear least-square problem. Solving the non-linear least-square problem is computationally expensive, proportional to the number of cameras, points, and projections. In this paper, we implement the Bundle Adjustment (BA) algorithm and analyze techniques to improve algorithmic performance by reducing the mean square error. We investigate using an additional radial distortion camera parameter in the BA algorithm and demonstrate better convergence of the mean square error. We also demonstrate the use of explicitly computed analytical derivatives. In addition, we implement the BA algorithm on GPUs using the CUDA parallel programming model to reduce the computational time burden of the BA algorithm. CUDA Streams, atomic operations, and cuBLAS library in the CUDA programming model are proposed, implemented, and demonstrated to improve the performance of the BA algorithm. Our implementation has demonstrated better convergence of the BA algorithm and achieved a speedup of up to 16× on the use of the BA algorithm on various datasets.

Vision based terrain reconstruction for planet rover using a special binocular bundle adjustment

This paper presents a pure vision based technique for 3D reconstruction of planet terrain. The reconstruction accuracy depends ultimately on an optimization technique known as 'bundle adjustment'. In vision techniques, the translation is only known up to a scale factor, and a single scale factor is assumed for the whole sequence of images if only one camera is used. If an extra camera is available, stereo vision based reconstruction can be obtained by binocular views. If the baseline of the stereo setup is known, the scale factor problem is solved. We found that direct application of classical bundle adjustment on the constraints inherent between the binocular views has not been tested. Our method incorporated this constraint into the conventional bundle adjustment method. This special binocular bundle adjustment has been performed on image sequences similar to planet terrain circumstances. Experimental results show that our special method enhances not only the localization accuracy, but also the terrain mapping quality.

Accuracy Analysis on Bundle Adjustment of Remote Sensing Images Based on Dual Quaternion

A bundle adjustment method of remote sensing images based on dual quaternion is presented,which conducted the uniform disposal corresponding location and attitude of sequence images by the dual quaternion.The constraint relationship of image itself and sequence images is constructed to compensate the systematic errors.The feasibility of this method used in bundle adjustment is theoretically tested by the analysis of the structural characteristics of error equation and normal equation based on dual quaternion.Different distributions of control points and stepwise regression analysis are introduced into the experiment for RC30 image.The results show that the adjustment accuracy can achieve 0.2min plane and 1min elevation.As a result,this method provides a new technique for geometric location problem of remote sensing images.

双基地ISAR运动补偿算法

在双基地ISAR成像模型的基础上,通过解线频调处理得出双基ISAR回波信号的差频域表示,针对参考信号跟踪存在误差而影响成像质量的问题,采用距离对准消除跟踪误差的影响;而后采用单特显点法补偿由于距离对准而产生的初相误差,最终给出了整个双基ISAR成像算法流程,最后通过对点目标模型的仿真,验证了该算法的有效性。

基于梯度下降法的ISAR最小熵相位校正算法

最小熵相位校正算法是一种非常有潜力的ISAR相位校正算法,有多种不同的方法可以实现最小熵相位校正算法,如定点迭代法即是一种典型的最小熵相位校正算法。文中运用梯度下降法来实现最小熵相位校正的方法,并且通过对实测数据的ISAR成像,将梯度下降法与其他方法的收敛速度进行了比较,同时讨论了如何对梯度下降法步长参数进行选择。

Distributed sparse bundle adjustment algorithm based on three-dimensional point partition and asynchronous communication

Sparse bundle adjustment(SBA) is a key but time-and memory-consuming step in three-dimensional(3 D) reconstruction. In this paper, we propose a 3 D point-based distributed SBA algorithm(DSBA) to improve the speed and scalability of SBA. The algorithm uses an asynchronously distributed sparse bundle adjustment(A-DSBA)to overlap data communication with equation computation. Compared with the synchronous DSBA mechanism(SDSBA), A-DSBA reduces the running time by 46%. The experimental results on several 3 D reconstruction datasets reveal that our distributed algorithm running on eight nodes is up to five times faster than that of the stand-alone parallel SBA. Furthermore, the speedup of the proposed algorithm(running on eight nodes with 48 cores) is up to41 times that of the serial SBA(running on a single node).

Hybrid Parallel Bundle Adjustment for 3D Scene Reconstruction with Massive Points

Bundle adjustment （BA） is a crucial but time consuming step in 3D reconstruction. In this paper, we intend to tackle a special class of BA problems where the reconstructed 3D points are much more numerous than the camera parameters, called Massive-Points BA （MPBA） problems. This is often the case when high-resolution images are used. We present a design and implementation of a new bundle adjustment algorithm for efficiently solving the MPBA problems. The use of hardware parallelism, the multi-core CPUs as well as GPUs, is explored. By careful memory-usage design, the graphic-memory limitation is effectively alleviated. Several modern acceleration strategies for bundle adjustment, such as the mixed-precision arithmetics, the embedded point iteration, and the preconditioned conjugate gradients, are explored and compared. By using several high-resolution image datasets, we generate a variety of MFBA problems, with which the performance of five bundle adjustment algorithms are evaluated. The experimental results show that our algorithm is up to 40 times faster than classical Sparse Bundle Adjustment, while maintaining comparable precision.

A low-overhead asynchronous consensus framework for distributed bundle adjustment

Generally,the distributed bundle adjustment(DBA)method uses multiple worker nodes to solve the bundle adjustment problems and overcomes the computation and memory storage limitations of a single computer.However,the performance considerably degrades owing to the overhead introduced by the additional block partitioning step and synchronous waiting.Therefore,we propose a low-overhead consensus framework.A partial barrier based asynchronous method is proposed to early achieve consensus with respect to the faster worker nodes to avoid waiting for the slower ones.A scene summarization procedure is designed and integrated into the block partitioning step to ensure that clustering can be performed on the small summarized scene.Experiments conducted on public datasets show that our method can improve the worker node utilization rate and reduce the block partitioning time.Also,sample applications are demonstrated using our large-scale culture heritage datasets.

BUNDLE ADJUSTMENTS CCD CAMERA CALIBRATION BASED ON COLLI NEARITY EQUATION

The solid template CCD camera calibration method of bundle adjustments basedon collinearity equation is presented considering the characteristics of space large-dimensionon-line measurement. In the method, a more comprehensive camera model is adopted which is based onthe pinhole model extended with distortions corrections. In the process of calibration, calibrationprecision is improved by imaging at different locations in the whole measurement space,multi-imaging at the same location and bundle adjustments optimization. The calibration experimentproves that the calibration method is able to fulfill calibration requirement of CCD camera appliedto vision measurement.

Bundle adjustment for data processing of theodolite industrial surveying system

The photogrammetric bundle adjustment was used in data processing of electronic theodolite industrial surveying system by converting angular observations into virtual photo coordinates. The developed algorithm has ability of precision estimation and data snooping, do not need initial values of exterior orientation elements and object point coordinates. The form of control condition for the system is quite flexible. Neither centering nor leveling is the theodolite needed and the lay out of theodolite position is flexible when the system is used for precise survey. Experiments carried out in test field verify the validity of the data processing method. [

附加距离约束的导航相机快速标定方法

针对在轨条件下因观测条件匮乏导致巡视器导航相机标定精度受限的问题,提出了一种基于直线特征的导航相机快速在轨自标定方法。首先,利用SAM(segment anything model)对导航影像太阳能板区域进行图像分割,并对太阳能板OCR(optical control room)直线特征进行提取;然后,构建附加距离约束的导航相机自检校模型;最后,利用Levenberg-Marquardt方法对标定模型进行求解。开展了模拟实验及在轨实验。分析结果表明,相比光束法自标定、CAHVOR及Vanishing Points等标定方法,所提出方法标定参数重建的坐标精度平均值分别提升了20.60%、30.82%和35.92%,对太阳能板OCR距离恢复距离偏差范围在-1.4~1.5 mm之间。该方法可为巡视器导航相机在轨标定技术提供一定的价值参考。

融合视惯RTK的非接触高精度点位测量方法

为解决信号受遮挡或物理不可达区域的高精度点位测量问题,提出了一种融合视惯和实时差分定位(RTK)信息的非接触高精度点位测量方法。首先,利用由RTK测定的相机位置信息,在增量式重建中将相机位姿和特征点空间位置配准到全球地理坐标系下;其次,在全局优化中引入视觉重投影误差、RTK测定的相机位置、惯性测量单元测定的相机姿态作为约束,以提升特征点空间位置的测量精度;最后,将图像中的特征点坐标从像平面坐标系映射到全球地理坐标系下,实现对场景中特征点位坐标的非接触性测量。与OpenMVG的对比实验表明:所提方法在3m和5m距离处的非接触性点位测量精度分别提高了87.25%和75.88%,验证了其在非接触性点位测量问题上的高精度。

基于窗口化匹配估计的ORB-SLAM算法研究

针对ORB-SLAM系统中求解相机位姿存在随机性而引起定位精度不稳定的问题,提出一种基于特征点窗口化匹配和解析ICP的改进位姿求解算法,构建移动机器人ORB-SLAM系统。通过对提取的特征点进行窗口化,在保证选取良好特征点匹配的同时提高匹配效率;利用具有解析解的ICP算法求解相机位姿避免迭代,选取误差最小的窗口位姿求解结果进行光束法平差优化,以减小局部信息丢失或误匹配引起的位姿误差。结果表明:改进算法能够提高相机轨迹求解精度,轨迹误差相比ORB-SLAM2和ORB-SLAM3平均减少30%以上,并可以降低相机跟踪失败的概率。

Impact of a Bundle on Prevention and Control of Healthcare associated Infections in Intensive Care Unit

Inpatients in the intensive care unit（ICU） are at high risk for healthcare-associated infections（HAIs）. In the current study, a bundle of interventions and measures for preventing and controlling HAIs were developed and implemented in the ICU by trained personnel, and the impact of the bundle was evaluated. The incidence of HAIs, the adjusted daily incidence of HAIs and the incidence of three types of catheter-related infections before and after the bundle implementation were compared. The execution rate of the bundle for preventing and controlling ventilator-associated pneumonia（VAP） was increased from 82.06% in 2012 to 96.88% in 2013. The execution rate was increased from 83.03% in 2012 to 91.33% in 2013 for central line-associated bloodstream infection（CLABSI）, from 87.00% to 94.40% for catheter-associated urinary tract infection（CAUTI）, and from 82.05% to 98.55% for multidrug-resistant organisms（MDROs）, respectively. In total, 136 cases（10.37%） in 2012 and 113 cases（7.72%） in 2013 involved HAIs, respectively. Patients suffered from infection of the lower respiratory tract, the most common site of HAIs, in 134 cases（79.29%） in 2012 and 107 cases（74.30%） in 2013 respectively. The incidence of VAP was 32.72‰ and 24.60‰, the number of strains of pathogens isolated was 198 and 173, and the number of MDROs detected in the ICU was 91 and 74 in 2012 and 2013, respectively. The percentage of MDROs among the pathogens causing HAIs was decreased in each quarter of 2013 as compared with the corresponding percentage in 2012. In 2013, the execution rate of the bundle for preventing and controlling HAIs was increased, whereas the incidence of HAIs and VAP decreased as compared with that in 2012.

基于开源GIS数据的广域小重叠卫星影像定位模型精化

设计了一种适用于大区域、小重叠卫星影像有理多项式模型(RPC)定位精度优化的技术流程,基于开源数字正射影像(DOM)、数字高程模型(DEM)数据和RPC几何模型约束,通过优化尺度不变特征变换(SIFT)特征提取与匹配算法,实现了无像幅约束的控制点和连接点的快速稳健匹配,应用DEM支持下的RPC模型二维区域网方法实现了多景影像RPC模型精度的联合精化。GF-1卫星WFV影像的实验结果表明:在DEM支持下经过区域网平差后,RPC模型的定位精度可达到像素级水平。

高分十四号立体测绘卫星无控定位精度初步评估

高分十四号卫星是我国目前测绘精度最高的卫星之一,采用了先进的多载荷一体化对地观测技术,主要用于全球范围内高精度定位和测制1∶10 000比例尺地理信息产品。本文重点介绍高分十四号卫星有效载荷和地面测绘处理系统及其性能,并利用国内外高精度检测场进行了几何性能评估。试验结果表明:单航线无控定位精度境内区域达到平面1.80 m(RMS)、高程0.80 m(RMS),境外区域达到平面1.76 m(RMS)、高程0.82 m(RMS),无控定位精度处于国际已公布的光学摄影测量卫星的最高水平。

三维摄影条件下视差角光束法平差模型的适用性研究

为了论证视差角光束法平差模型在短基线摄影条件下的适用性,将视差角光束法平差(PBA)模型中基于二维假设的数学证明扩展到三维,研究视差角参数对观测噪声的敏感性、法方程奇异性以及线性化程度,并基于2.11×10^(-8)~2.11×10^(-12)弧度的小交会角的短基线摄影条件进行仿真和真实实验。理论分析和实验结果表明,目前的PBA模型仅适用于二维摄影,不能解决三维摄影条件下短基线的平差问题。

基于视觉辅助优化的惯性可通行地图构建与行人定位方法

针对以惯性传感器为核心的室内行人导航系统存在长时间高精准定位问题,在基于历史信息约束的行人惯性定位方法基础上,提出了一种基于视觉辅助优化的惯性可通行地图构建与行人定位方法。以惯性传感器为核心,引入低频视觉信息优化行人轨迹,在滤波框架下加入优化线程。在粒子滤波的基础上,提取图像的VLAD特征作为地标特征,结合位置信息与地标信息进行地标特征匹配,根据匹配结果采用光束平差法基于拓扑结构优化行人轨迹,最终通过粒子重采样实现位置误差修正。在总长度1450 m的楼宇行走实验中,与基于历史信息约束的行人惯性定位方法相比,所提方法的平均定位误差为1.50 m,定位精度提升约40%。