Optimized CUDA Implementation to Improve the Performance of Bundle Adjustment Algorithm on GPUs

The 3D reconstruction pipeline uses the Bundle Adjustment algorithm to refine the camera and point parameters. The Bundle Adjustment algorithm is a compute-intensive algorithm, and many researchers have improved its performance by implementing the algorithm on GPUs. In the previous research work, “Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm using GPUs,” the authors demonstrated first the Bundle Adjustment algorithmic performance improvement by reducing the mean square error using an additional radial distorting parameter and explicitly computed analytical derivatives and reducing the computational burden of the Bundle Adjustment algorithm using GPUs. The naïve implementation of the CUDA code, a speedup of 10× for the largest dataset of 13,678 cameras, 4,455,747 points, and 28,975,571 projections was achieved. In this paper, we present the optimization of the Bundle Adjustment algorithm CUDA code on GPUs to achieve higher speedup. We propose a new data memory layout for the parameters in the Bundle Adjustment algorithm, resulting in contiguous memory access. We demonstrate that it improves the memory throughput on the GPUs, thereby improving the overall performance. We also demonstrate an increase in the computational throughput of the algorithm by optimizing the CUDA kernels to utilize the GPU resources effectively. A comparative performance study of explicitly computing an algorithm parameter versus using the Jacobians instead is presented. In the previous work, the Bundle Adjustment algorithm failed to converge for certain datasets due to several block matrices of the cameras in the augmented normal equation, resulting in rank-deficient matrices. In this work, we identify the cameras that cause rank-deficient matrices and preprocess the datasets to ensure the convergence of the BA algorithm. Our optimized CUDA implementation achieves convergence of the Bundle Adjustment algorithm in around 22 seconds for the largest dataset compared to 654 seconds for the sequential implementation, resulting in a speedup of 30×. Our optimized CUDA implementation presented in this paper has achieved a 3× speedup for the largest dataset compared to the previous naïve CUDA implementation.

A Self-calibration Bundle Adjustment Algorithm Based on Block Matrix Cholesky Decomposition Technology

In this study,the problem of bundle adjustment was revisited,and a novel algorithm based on block matrix Cholesky decomposition was proposed to solve the thorny problem of self-calibration bundle adjustment.The innovation points are reflected in the following aspects:①The proposed algorithm is not dependent on the Schur complement,and the calculation process is simple and clear;②The complexities of time and space tend to O(n)in the context of world point number is far greater than that of images and cameras,so the calculation magnitude and memory consumption can be reduced significantly;③The proposed algorithm can carry out self-calibration bundle adjustment in single-camera,multi-camera,and variable-camera modes;④Some measures are employed to improve the optimization effects.Experimental tests showed that the proposed algorithm has the ability to achieve state-of-the-art performance in accuracy and robustness,and it has a strong adaptability as well,because the optimized results are accurate and robust even if the initial values have large deviations from the truth.This study could provide theoretical guidance and technical support for the image-based positioning and 3D reconstruction in the fields of photogrammetry,computer vision and robotics.

Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm Using GPUs

Bundle adjustment is a camera and point refinement technique in a 3D scene reconstruction pipeline. The camera parameters and the 3D points are refined by minimizing the difference between computed projection and observed projection of the image points formulated as a non-linear least-square problem. Levenberg-Marquardt method is used to solve the non-linear least-square problem. Solving the non-linear least-square problem is computationally expensive, proportional to the number of cameras, points, and projections. In this paper, we implement the Bundle Adjustment (BA) algorithm and analyze techniques to improve algorithmic performance by reducing the mean square error. We investigate using an additional radial distortion camera parameter in the BA algorithm and demonstrate better convergence of the mean square error. We also demonstrate the use of explicitly computed analytical derivatives. In addition, we implement the BA algorithm on GPUs using the CUDA parallel programming model to reduce the computational time burden of the BA algorithm. CUDA Streams, atomic operations, and cuBLAS library in the CUDA programming model are proposed, implemented, and demonstrated to improve the performance of the BA algorithm. Our implementation has demonstrated better convergence of the BA algorithm and achieved a speedup of up to 16× on the use of the BA algorithm on various datasets.

BUNDLE ADJUSTMENTS CCD CAMERA CALIBRATION BASED ON COLLI NEARITY EQUATION

The solid template CCD camera calibration method of bundle adjustments basedon collinearity equation is presented considering the characteristics of space large-dimensionon-line measurement. In the method, a more comprehensive camera model is adopted which is based onthe pinhole model extended with distortions corrections. In the process of calibration, calibrationprecision is improved by imaging at different locations in the whole measurement space,multi-imaging at the same location and bundle adjustments optimization. The calibration experimentproves that the calibration method is able to fulfill calibration requirement of CCD camera appliedto vision measurement.

Bundle adjustment for data processing of theodolite industrial surveying system

The photogrammetric bundle adjustment was used in data processing of electronic theodolite industrial surveying system by converting angular observations into virtual photo coordinates. The developed algorithm has ability of precision estimation and data snooping, do not need initial values of exterior orientation elements and object point coordinates. The form of control condition for the system is quite flexible. Neither centering nor leveling is the theodolite needed and the lay out of theodolite position is flexible when the system is used for precise survey. Experiments carried out in test field verify the validity of the data processing method. [

Hybrid Parallel Bundle Adjustment for 3D Scene Reconstruction with Massive Points

Bundle adjustment （BA） is a crucial but time consuming step in 3D reconstruction. In this paper, we intend to tackle a special class of BA problems where the reconstructed 3D points are much more numerous than the camera parameters, called Massive-Points BA （MPBA） problems. This is often the case when high-resolution images are used. We present a design and implementation of a new bundle adjustment algorithm for efficiently solving the MPBA problems. The use of hardware parallelism, the multi-core CPUs as well as GPUs, is explored. By careful memory-usage design, the graphic-memory limitation is effectively alleviated. Several modern acceleration strategies for bundle adjustment, such as the mixed-precision arithmetics, the embedded point iteration, and the preconditioned conjugate gradients, are explored and compared. By using several high-resolution image datasets, we generate a variety of MFBA problems, with which the performance of five bundle adjustment algorithms are evaluated. The experimental results show that our algorithm is up to 40 times faster than classical Sparse Bundle Adjustment, while maintaining comparable precision.

A low-overhead asynchronous consensus framework for distributed bundle adjustment

Generally,the distributed bundle adjustment(DBA)method uses multiple worker nodes to solve the bundle adjustment problems and overcomes the computation and memory storage limitations of a single computer.However,the performance considerably degrades owing to the overhead introduced by the additional block partitioning step and synchronous waiting.Therefore,we propose a low-overhead consensus framework.A partial barrier based asynchronous method is proposed to early achieve consensus with respect to the faster worker nodes to avoid waiting for the slower ones.A scene summarization procedure is designed and integrated into the block partitioning step to ensure that clustering can be performed on the small summarized scene.Experiments conducted on public datasets show that our method can improve the worker node utilization rate and reduce the block partitioning time.Also,sample applications are demonstrated using our large-scale culture heritage datasets.

Impact of a Bundle on Prevention and Control of Healthcare associated Infections in Intensive Care Unit

Inpatients in the intensive care unit（ICU） are at high risk for healthcare-associated infections（HAIs）. In the current study, a bundle of interventions and measures for preventing and controlling HAIs were developed and implemented in the ICU by trained personnel, and the impact of the bundle was evaluated. The incidence of HAIs, the adjusted daily incidence of HAIs and the incidence of three types of catheter-related infections before and after the bundle implementation were compared. The execution rate of the bundle for preventing and controlling ventilator-associated pneumonia（VAP） was increased from 82.06% in 2012 to 96.88% in 2013. The execution rate was increased from 83.03% in 2012 to 91.33% in 2013 for central line-associated bloodstream infection（CLABSI）, from 87.00% to 94.40% for catheter-associated urinary tract infection（CAUTI）, and from 82.05% to 98.55% for multidrug-resistant organisms（MDROs）, respectively. In total, 136 cases（10.37%） in 2012 and 113 cases（7.72%） in 2013 involved HAIs, respectively. Patients suffered from infection of the lower respiratory tract, the most common site of HAIs, in 134 cases（79.29%） in 2012 and 107 cases（74.30%） in 2013 respectively. The incidence of VAP was 32.72‰ and 24.60‰, the number of strains of pathogens isolated was 198 and 173, and the number of MDROs detected in the ICU was 91 and 74 in 2012 and 2013, respectively. The percentage of MDROs among the pathogens causing HAIs was decreased in each quarter of 2013 as compared with the corresponding percentage in 2012. In 2013, the execution rate of the bundle for preventing and controlling HAIs was increased, whereas the incidence of HAIs and VAP decreased as compared with that in 2012.

3D motion and geometric information system of single-antenna radar based on incomplete 1D range data

A 3D motion and geometric information system of single-antenna radar is proposed,which can be supported by spotlight synthetic aperture radar（SAR） system and inverse SAR（ISAR） system involving relative 3D motion of the rigid target.In this system,applying the geometry invariance of the rigid target,the unknown 3D shape and motion of the radar target can be reconstructed from the 1D range data of some scatterers extracted from the high-resolution range image.Compared with the current 1D-to-3D algorithm,in the proposed algorithm,the requirement of the 1D range data is expanded to incomplete formation involving large angular motion of the target and hence,the quantity of the scatterers and the abundance of 3D motion are enriched.Furthermore,with the three selected affine coordinates fixed,the multi-solution problem of the reconstruction is solved and the technique of nonlinear optimization can be successfully utilized in the system.Two simulations are implemented which verify the higher robustness of the system and the better performance of the 3D reconstruction for the radar target with unknown relative motion.

A preliminary study on photogrammetry for the FAST main reflector measurement

The Five-hundred-meter Aperture Spherical radio Telescope(FAST) is the largest single-dish radio telescope in the world. In this paper, we apply photogrammetry to measure local area surface accuracy of the FAST main reflector. Contrast with the existing photogrammetric methods that need to stick the cooperation target on the panel. In this paper, we directly detect nodes according to their natural feature. Analyzing the FAST reflector composition, we propose a two-step Hough transform method for node detection. Due to the high similarity of the neighboring area around nodes, it is hard to match two images by the feature matching method. Therefore, we apply the nodes combination approach to obtain the homography matrix between two photos for nodes matching. After nodes detection and matching,triangulation and bundle adjustment algorithms are adopted for 3 D reconstruction. During the experiment,the adjusted node position deform a local area of the FAST main reflector into a spherical cap with a radius of 300 m. The experimental result shows that the measurement accuracy of the sphere with a radius of 300 m is 12.299 mm, indicating that it is feasible to apply photogrammetry for the FAST main reflector measurement.

Flexible Planar-Scene Camera Calibration Technique

A flexible camera calibration technique using 2D-DLT and bundle adjustment with planar scenes is proposed. The equation of principal line under image coordinate system represented with 2D-DLT parameters is educed using the correspondence between collinearity equations and 2D-DLT. A novel algorithm to obtain the initial value of principal point is put forward. Proof of Critical Motion Sequences for calibration is given in detail. The practical decomposition algorithm of exterior parameters using initial values of principal point, focal length and 2D-DLT parameters is discussed elaborately. Planar\|scene camera calibration algorithm with bundle adjustment is addressed. Very good results have been obtained with both computer simulations and real data calibration. The calibration result can be used in some high precision applications, such as reverse engineering and industrial inspection.

3-D Free-form Shape Measuring System Using Unconstrained Range Sensor

Three-dimensional（3-D） free-form shape measurement,a challenging task pursued by computer vision,is mainly characterized with single view acquisition and multiple view registration.Most of the conventional scanning systems are less flexibility and difficult to realize engineering applications for employing sequential registration tactic.To develop portable scanning system and engineering registration method overcoming problems of error accumulation and propagation is the research direction.In this paper,one 3-D free-form shape measuring system using unconstrained range sensor is designed.A quasi-active stereo binocular visual sensor embedded within a scanning mechanism is used as the range sensor.Error compensation is performed by residual amendment according to camera calibration lattice.Artificial control points are designed and adhered on object and one camera is introduced to shot these control points from different positions and orientations.Then ray bundle adjustment（BA） method is used to calculate the space coordinates of all the control points,so as to set up a global control net work.Registration can be completed by mapping at least 3 control points observed by range sensor in single view acquisition into the global control network.In this system,no calibration for laser plane is required and the motion of range sensor is completely free.The overlapping of neighboring region is unessential for registration.Therefore,the working range of the system can be easily extended.The measuring precision mainly depends on the quality of global control network.The sequential distances of coding control points are observed by electronic theodolites and then compared with those obtained according to BA result.Experimental results show that relative distance error of control points is no more than 0.2%.The proposed measuring system is portable,provides good capacity for global error control,and contributes to the engineering application of 3-D free-form shape measurement.

KLT-VIO:Real-time Monocular Visual-Inertial Odometry

This paper proposes a Visual-Inertial Odometry(VIO)algorithm that relies solely on monocular cameras and Inertial Measurement Units(IMU),capable of real-time self-position estimation for robots during movement.By integrating the optical flow method,the algorithm tracks both point and line features in images simultaneously,significantly reducing computational complexity and the matching time for line feature descriptors.Additionally,this paper advances the triangulation method for line features,using depth information from line segment endpoints to determine their Plcker coordinates in three-dimensional space.Tests on the EuRoC datasets show that the proposed algorithm outperforms PL-VIO in terms of processing speed per frame,with an approximate 5%to 10%improvement in both relative pose error(RPE)and absolute trajectory error(ATE).These results demonstrate that the proposed VIO algorithm is an efficient solution suitable for low-computing platforms requiring real-time localization and navigation.