A Self-calibration Bundle Adjustment Algorithm Based on Block Matrix Cholesky Decomposition Technology

In this study,the problem of bundle adjustment was revisited,and a novel algorithm based on block matrix Cholesky decomposition was proposed to solve the thorny problem of self-calibration bundle adjustment.The innovation points are reflected in the following aspects:①The proposed algorithm is not dependent on the Schur complement,and the calculation process is simple and clear;②The complexities of time and space tend to O(n)in the context of world point number is far greater than that of images and cameras,so the calculation magnitude and memory consumption can be reduced significantly;③The proposed algorithm can carry out self-calibration bundle adjustment in single-camera,multi-camera,and variable-camera modes;④Some measures are employed to improve the optimization effects.Experimental tests showed that the proposed algorithm has the ability to achieve state-of-the-art performance in accuracy and robustness,and it has a strong adaptability as well,because the optimized results are accurate and robust even if the initial values have large deviations from the truth.This study could provide theoretical guidance and technical support for the image-based positioning and 3D reconstruction in the fields of photogrammetry,computer vision and robotics.

Optimized CUDA Implementation to Improve the Performance of Bundle Adjustment Algorithm on GPUs

The 3D reconstruction pipeline uses the Bundle Adjustment algorithm to refine the camera and point parameters. The Bundle Adjustment algorithm is a compute-intensive algorithm, and many researchers have improved its performance by implementing the algorithm on GPUs. In the previous research work, “Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm using GPUs,” the authors demonstrated first the Bundle Adjustment algorithmic performance improvement by reducing the mean square error using an additional radial distorting parameter and explicitly computed analytical derivatives and reducing the computational burden of the Bundle Adjustment algorithm using GPUs. The naïve implementation of the CUDA code, a speedup of 10× for the largest dataset of 13,678 cameras, 4,455,747 points, and 28,975,571 projections was achieved. In this paper, we present the optimization of the Bundle Adjustment algorithm CUDA code on GPUs to achieve higher speedup. We propose a new data memory layout for the parameters in the Bundle Adjustment algorithm, resulting in contiguous memory access. We demonstrate that it improves the memory throughput on the GPUs, thereby improving the overall performance. We also demonstrate an increase in the computational throughput of the algorithm by optimizing the CUDA kernels to utilize the GPU resources effectively. A comparative performance study of explicitly computing an algorithm parameter versus using the Jacobians instead is presented. In the previous work, the Bundle Adjustment algorithm failed to converge for certain datasets due to several block matrices of the cameras in the augmented normal equation, resulting in rank-deficient matrices. In this work, we identify the cameras that cause rank-deficient matrices and preprocess the datasets to ensure the convergence of the BA algorithm. Our optimized CUDA implementation achieves convergence of the Bundle Adjustment algorithm in around 22 seconds for the largest dataset compared to 654 seconds for the sequential implementation, resulting in a speedup of 30×. Our optimized CUDA implementation presented in this paper has achieved a 3× speedup for the largest dataset compared to the previous naïve CUDA implementation.

Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm Using GPUs

Bundle adjustment is a camera and point refinement technique in a 3D scene reconstruction pipeline. The camera parameters and the 3D points are refined by minimizing the difference between computed projection and observed projection of the image points formulated as a non-linear least-square problem. Levenberg-Marquardt method is used to solve the non-linear least-square problem. Solving the non-linear least-square problem is computationally expensive, proportional to the number of cameras, points, and projections. In this paper, we implement the Bundle Adjustment (BA) algorithm and analyze techniques to improve algorithmic performance by reducing the mean square error. We investigate using an additional radial distortion camera parameter in the BA algorithm and demonstrate better convergence of the mean square error. We also demonstrate the use of explicitly computed analytical derivatives. In addition, we implement the BA algorithm on GPUs using the CUDA parallel programming model to reduce the computational time burden of the BA algorithm. CUDA Streams, atomic operations, and cuBLAS library in the CUDA programming model are proposed, implemented, and demonstrated to improve the performance of the BA algorithm. Our implementation has demonstrated better convergence of the BA algorithm and achieved a speedup of up to 16× on the use of the BA algorithm on various datasets.

BUNDLE ADJUSTMENTS CCD CAMERA CALIBRATION BASED ON COLLI NEARITY EQUATION

The solid template CCD camera calibration method of bundle adjustments basedon collinearity equation is presented considering the characteristics of space large-dimensionon-line measurement. In the method, a more comprehensive camera model is adopted which is based onthe pinhole model extended with distortions corrections. In the process of calibration, calibrationprecision is improved by imaging at different locations in the whole measurement space,multi-imaging at the same location and bundle adjustments optimization. The calibration experimentproves that the calibration method is able to fulfill calibration requirement of CCD camera appliedto vision measurement.

Vision based terrain reconstruction for planet rover using a special binocular bundle adjustment

This paper presents a pure vision based technique for 3D reconstruction of planet terrain. The reconstruction accuracy depends ultimately on an optimization technique known as 'bundle adjustment'. In vision techniques, the translation is only known up to a scale factor, and a single scale factor is assumed for the whole sequence of images if only one camera is used. If an extra camera is available, stereo vision based reconstruction can be obtained by binocular views. If the baseline of the stereo setup is known, the scale factor problem is solved. We found that direct application of classical bundle adjustment on the constraints inherent between the binocular views has not been tested. Our method incorporated this constraint into the conventional bundle adjustment method. This special binocular bundle adjustment has been performed on image sequences similar to planet terrain circumstances. Experimental results show that our special method enhances not only the localization accuracy, but also the terrain mapping quality.

Accuracy Analysis on Bundle Adjustment of Remote Sensing Images Based on Dual Quaternion

A bundle adjustment method of remote sensing images based on dual quaternion is presented,which conducted the uniform disposal corresponding location and attitude of sequence images by the dual quaternion.The constraint relationship of image itself and sequence images is constructed to compensate the systematic errors.The feasibility of this method used in bundle adjustment is theoretically tested by the analysis of the structural characteristics of error equation and normal equation based on dual quaternion.Different distributions of control points and stepwise regression analysis are introduced into the experiment for RC30 image.The results show that the adjustment accuracy can achieve 0.2min plane and 1min elevation.As a result,this method provides a new technique for geometric location problem of remote sensing images.

Bundle adjustment for data processing of theodolite industrial surveying system

The photogrammetric bundle adjustment was used in data processing of electronic theodolite industrial surveying system by converting angular observations into virtual photo coordinates. The developed algorithm has ability of precision estimation and data snooping, do not need initial values of exterior orientation elements and object point coordinates. The form of control condition for the system is quite flexible. Neither centering nor leveling is the theodolite needed and the lay out of theodolite position is flexible when the system is used for precise survey. Experiments carried out in test field verify the validity of the data processing method. [

Impact of a Bundle on Prevention and Control of Healthcare associated Infections in Intensive Care Unit

Inpatients in the intensive care unit（ICU） are at high risk for healthcare-associated infections（HAIs）. In the current study, a bundle of interventions and measures for preventing and controlling HAIs were developed and implemented in the ICU by trained personnel, and the impact of the bundle was evaluated. The incidence of HAIs, the adjusted daily incidence of HAIs and the incidence of three types of catheter-related infections before and after the bundle implementation were compared. The execution rate of the bundle for preventing and controlling ventilator-associated pneumonia（VAP） was increased from 82.06% in 2012 to 96.88% in 2013. The execution rate was increased from 83.03% in 2012 to 91.33% in 2013 for central line-associated bloodstream infection（CLABSI）, from 87.00% to 94.40% for catheter-associated urinary tract infection（CAUTI）, and from 82.05% to 98.55% for multidrug-resistant organisms（MDROs）, respectively. In total, 136 cases（10.37%） in 2012 and 113 cases（7.72%） in 2013 involved HAIs, respectively. Patients suffered from infection of the lower respiratory tract, the most common site of HAIs, in 134 cases（79.29%） in 2012 and 107 cases（74.30%） in 2013 respectively. The incidence of VAP was 32.72‰ and 24.60‰, the number of strains of pathogens isolated was 198 and 173, and the number of MDROs detected in the ICU was 91 and 74 in 2012 and 2013, respectively. The percentage of MDROs among the pathogens causing HAIs was decreased in each quarter of 2013 as compared with the corresponding percentage in 2012. In 2013, the execution rate of the bundle for preventing and controlling HAIs was increased, whereas the incidence of HAIs and VAP decreased as compared with that in 2012.

附加距离约束的导航相机快速标定方法

针对在轨条件下因观测条件匮乏导致巡视器导航相机标定精度受限的问题,提出了一种基于直线特征的导航相机快速在轨自标定方法。首先,利用SAM(segment anything model)对导航影像太阳能板区域进行图像分割,并对太阳能板OCR(optical control room)直线特征进行提取;然后,构建附加距离约束的导航相机自检校模型;最后,利用Levenberg-Marquardt方法对标定模型进行求解。开展了模拟实验及在轨实验。分析结果表明,相比光束法自标定、CAHVOR及Vanishing Points等标定方法,所提出方法标定参数重建的坐标精度平均值分别提升了20.60%、30.82%和35.92%,对太阳能板OCR距离恢复距离偏差范围在-1.4~1.5 mm之间。该方法可为巡视器导航相机在轨标定技术提供一定的价值参考。

融合视惯RTK的非接触高精度点位测量方法

为解决信号受遮挡或物理不可达区域的高精度点位测量问题,提出了一种融合视惯和实时差分定位(RTK)信息的非接触高精度点位测量方法。首先,利用由RTK测定的相机位置信息,在增量式重建中将相机位姿和特征点空间位置配准到全球地理坐标系下;其次,在全局优化中引入视觉重投影误差、RTK测定的相机位置、惯性测量单元测定的相机姿态作为约束,以提升特征点空间位置的测量精度;最后,将图像中的特征点坐标从像平面坐标系映射到全球地理坐标系下,实现对场景中特征点位坐标的非接触性测量。与OpenMVG的对比实验表明:所提方法在3m和5m距离处的非接触性点位测量精度分别提高了87.25%和75.88%,验证了其在非接触性点位测量问题上的高精度。

光学运动捕捉实验中参考相机的选择方法研究

针对运动捕捉实验过程中多目视觉系统的标定优化问题,通过对摄像机参数估计过程的误差分析,探索了参考相机的选择对于光束平差迭代计算的影响,并在此基础上提出了一种Q值评测方法,该方法运用有权无向图获得相机节点之间的最短连通路径,利用反投影误差和均方差的加权平均值来评估参考相机选择的合理性。在实验中,使用了多台摄像机组成多目视觉系统对所提出的参考相机选择方法进行了验证。实验结果表明,所提出的Q值评测方法是有效的,可进一步提高系统标定的精度,对运动测试实验精度控制具有重要的参考价值。

无人机热红外影像快速拼接的改进SFM算法研究

无人机热红外成像技术在场景监控、物体识别等领域应用广泛,但是该影像具有分辨率低、倾角大、画幅小等缺点,因此无人机热红外影像拼接比较困难。本文提出了一种改进SFM(运动恢复结构)的无人机热红外影像的快速拼接算法(FastSFM)。FastSFM利用无人机热红外影像序列具有的时空顺序性,采用串联式影像匹配代替整体影像两两匹配,再根据稀疏矩阵的特性对Levenberg-Marquardt光束法平差方法进行优化,利用线性公式代替矩阵运算,从而提高拼接效率。通过实地采集影像数据进行了SFM和FastSFM拼接对比实验,20张热红外影像拼接时间从521 s提高到39 s,50张红外影像拼接时间从3 102 s提高到58 s,且拼接精度也有一定提高。结果表明,FastSFM能够实现无人机热红外影像的快速拼接。

视觉SLAM机器人中光束法平差优化芯片研究综述

在视觉同时定位与地图构建(Visual Simultaneous Localization and Mapping,V SLAM)系统中,光束法平差(Bundle Adjustment,BA)是优化相机参数和三维点位置的重要环节。然而,由于BA计算复杂度高,实时性要求高,传统的计算平台难以满足高效计算的需求。近年来,专用硬件加速器的引入为BA优化提供了新的解决方案。本文综述了BA优化专用芯片的研究现状及发展趋势,主要涵盖了BA算法的应用场景、定义与基本原理;BA在现场可编程门阵列(Field Programmable Gate Arrays,FPGA)、专用集成电路(Application Specific Integrated Circuits,ASIC)和图形处理单元(Graphics Processing Units,GPU)上的加速方法,以及这些加速器的发展趋势。此外,本文还探讨了BA加速器在技术实现中面临的挑战,并展望了其未来的发展方向。

基于窗口化匹配估计的ORB-SLAM算法研究

针对ORB-SLAM系统中求解相机位姿存在随机性而引起定位精度不稳定的问题,提出一种基于特征点窗口化匹配和解析ICP的改进位姿求解算法,构建移动机器人ORB-SLAM系统。通过对提取的特征点进行窗口化,在保证选取良好特征点匹配的同时提高匹配效率;利用具有解析解的ICP算法求解相机位姿避免迭代,选取误差最小的窗口位姿求解结果进行光束法平差优化,以减小局部信息丢失或误匹配引起的位姿误差。结果表明:改进算法能够提高相机轨迹求解精度,轨迹误差相比ORB-SLAM2和ORB-SLAM3平均减少30%以上,并可以降低相机跟踪失败的概率。

Hybrid Parallel Bundle Adjustment for 3D Scene Reconstruction with Massive Points

Bundle adjustment （BA） is a crucial but time consuming step in 3D reconstruction. In this paper, we intend to tackle a special class of BA problems where the reconstructed 3D points are much more numerous than the camera parameters, called Massive-Points BA （MPBA） problems. This is often the case when high-resolution images are used. We present a design and implementation of a new bundle adjustment algorithm for efficiently solving the MPBA problems. The use of hardware parallelism, the multi-core CPUs as well as GPUs, is explored. By careful memory-usage design, the graphic-memory limitation is effectively alleviated. Several modern acceleration strategies for bundle adjustment, such as the mixed-precision arithmetics, the embedded point iteration, and the preconditioned conjugate gradients, are explored and compared. By using several high-resolution image datasets, we generate a variety of MFBA problems, with which the performance of five bundle adjustment algorithms are evaluated. The experimental results show that our algorithm is up to 40 times faster than classical Sparse Bundle Adjustment, while maintaining comparable precision.

Distributed sparse bundle adjustment algorithm based on three-dimensional point partition and asynchronous communication

Sparse bundle adjustment(SBA) is a key but time-and memory-consuming step in three-dimensional(3 D) reconstruction. In this paper, we propose a 3 D point-based distributed SBA algorithm(DSBA) to improve the speed and scalability of SBA. The algorithm uses an asynchronously distributed sparse bundle adjustment(A-DSBA)to overlap data communication with equation computation. Compared with the synchronous DSBA mechanism(SDSBA), A-DSBA reduces the running time by 46%. The experimental results on several 3 D reconstruction datasets reveal that our distributed algorithm running on eight nodes is up to five times faster than that of the stand-alone parallel SBA. Furthermore, the speedup of the proposed algorithm(running on eight nodes with 48 cores) is up to41 times that of the serial SBA(running on a single node).

A low-overhead asynchronous consensus framework for distributed bundle adjustment

Generally,the distributed bundle adjustment(DBA)method uses multiple worker nodes to solve the bundle adjustment problems and overcomes the computation and memory storage limitations of a single computer.However,the performance considerably degrades owing to the overhead introduced by the additional block partitioning step and synchronous waiting.Therefore,we propose a low-overhead consensus framework.A partial barrier based asynchronous method is proposed to early achieve consensus with respect to the faster worker nodes to avoid waiting for the slower ones.A scene summarization procedure is designed and integrated into the block partitioning step to ensure that clustering can be performed on the small summarized scene.Experiments conducted on public datasets show that our method can improve the worker node utilization rate and reduce the block partitioning time.Also,sample applications are demonstrated using our large-scale culture heritage datasets.

基于开源GIS数据的广域小重叠卫星影像定位模型精化

设计了一种适用于大区域、小重叠卫星影像有理多项式模型(RPC)定位精度优化的技术流程,基于开源数字正射影像(DOM)、数字高程模型(DEM)数据和RPC几何模型约束,通过优化尺度不变特征变换(SIFT)特征提取与匹配算法,实现了无像幅约束的控制点和连接点的快速稳健匹配,应用DEM支持下的RPC模型二维区域网方法实现了多景影像RPC模型精度的联合精化。GF-1卫星WFV影像的实验结果表明:在DEM支持下经过区域网平差后,RPC模型的定位精度可达到像素级水平。

高分十四号立体测绘卫星无控定位精度初步评估

高分十四号卫星是我国目前测绘精度最高的卫星之一,采用了先进的多载荷一体化对地观测技术,主要用于全球范围内高精度定位和测制1∶10 000比例尺地理信息产品。本文重点介绍高分十四号卫星有效载荷和地面测绘处理系统及其性能,并利用国内外高精度检测场进行了几何性能评估。试验结果表明:单航线无控定位精度境内区域达到平面1.80 m(RMS)、高程0.80 m(RMS),境外区域达到平面1.76 m(RMS)、高程0.82 m(RMS),无控定位精度处于国际已公布的光学摄影测量卫星的最高水平。

三维摄影条件下视差角光束法平差模型的适用性研究

为了论证视差角光束法平差模型在短基线摄影条件下的适用性,将视差角光束法平差(PBA)模型中基于二维假设的数学证明扩展到三维,研究视差角参数对观测噪声的敏感性、法方程奇异性以及线性化程度,并基于2.11×10^(-8)~2.11×10^(-12)弧度的小交会角的短基线摄影条件进行仿真和真实实验。理论分析和实验结果表明,目前的PBA模型仅适用于二维摄影,不能解决三维摄影条件下短基线的平差问题。