An Iterative Pose Estimation Algorithm Based on Epipolar Geometry With Application to Multi-Target Tracking

This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit(IMU) is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking(MTT) in the presence of parallax and demonstrate its real-time performance on a 640 × 480 video sequence captured on a UAV. Video results are available at https://youtu.be/Hh K-p2 h XNn U.

Virtual Epipolar Line Construction of Single-Lens Bi-Prism Stereovision System

This paper proposes a simple geometrical ray based approach to solve the stereo correspondence problem for the single-lens bi-prism stereovision system. Each image captured using this system can be divided into two sub-images on the left and right and these sub-images are generated by two virtual cameras which are produced by the bi-prism. This stereovision system is equivalent to the conventional two camera system and the two sub-images captured have disparities which can be used to reconstruct back the 3-dimensional （3D） scene. The stereo correspondence problem of this system will be solved geometrically by applying the epipolar geometry constraint on the generated virtual cameras instead of the real CCD camera. Experiments are conducted to validate the proposed method and the results are compared to the calibration based approach to confirm its accuracy and effectiveness.

Feature matching based on geometric constraints in weakly calibrated stereo views of curved scenes

The identification of the correspondences of points of views is an important task. A new feature matching algorithm for weakly calibrated stereo images of curved scenes is proposed, based on mere geometric constraints. After initial correspondences are built via the epipolar constraint, many point-to-point image mappings called homographies are set up to predict the matching position for feature points. To refine the predictions and reject false correspondences, four schemes are proposed. Extensive experiments on simulated data as well as on real images of scenes of variant depths show that the proposed method is effective and robust.

View synthesis based on the serial images from camera lengthways motion

For the pre-acquired serial images from camera lengthways motion, a view synthesis algorithm based on epipolar geometry constraint is proposed in this paper. It uses the whole matching and maintaining order characters of the epipolar line, Fourier transform and dynamic programming matching theories, thus truly synthesizing the destination image of current viewpoint. Through the combination of Fourier transform, epipolar geometry constraint and dynamic programming matching, the circumference distortion problem resulting from conventional view synthesis approaches is effectively avoided. The detailed implementation steps of this algorithm are given, and some running instances are presented to illustrate the results.

Stereo Matching of Planar Curves Composed of Time Stamped Points

Matching features such as curve segments in stereo images play a very important role in scene recomtruction. In this paper, a stereo matching algorithm for the trajectories composed of time stamped points is proposed. Based on time stamped points, planar curve match measurements are given first, such as time constraint, cross-ratio invariant constraint and eplpolar geometry constraint; then, a trajectory matching method is proposed based on epipolar geometry constraint and cross-ratio invariant constraint. In order to match the planar curve segments projected by perspective projection system, the curve start time and end time are selected first to prepare match candidates. Then, the epipolar equation is used to discard the unmatched curve segment candidates. At last, a cross ratio invariant constxaint is used to find the most matched curve segments. If their match measurement is higher than the specialized threshold, a candidate with the least cross ratio difference is then selected as the match result; otherwise, no match is found. Unlike the conventional planar curve segments matching algorithm, this paper presents a weakly calibrated binocular stereo vision system which is based on wide baseline. The stamped points are obtained by targets detecting method of flying objects from image sequences. Due to wide baseline, there must exist the projection not in epipolar monotonic order or the curve segments located in very short distance and keeping the epipolar monotonic order. By using the method mentioned above, experiments are made to match planar curve segments not only in epipolar monotonic order but also not in epipolar monotonic order. The results show that the performance of our curve matching algorithm is effective for matching the arc-like planar trajectories composed of time stamped points.

Unwrapping and stereo rectification for omnidirectional images

Omnidirectional imaging sensors have been used in more and more applications when a very large field of view is required.In this paper,we investigate the unwrapping,epipolar geometry and stereo rectification issues for omnidirectional vision when the particular mirror model and the camera parameters are unknown in priori.First,the omnidirectional camera is calibrated under the Taylor model,and the parameters related to this model are obtained.In order to make the classical computer vision algorithms of conventional perspective cameras applicable,the ring omnidirectional image is unwrapped into two kinds of panoramas:cylinder and cuboid.Then the epipolar geometry of arbitrary camera configuration is analyzed and the essential matrix is deduced with its properties being indicated for ring images.After that,a simple stereo rectification method based on the essential matrix and the conformal mapping is proposed.Simulations and real data experimental results illustrate that our methods are effective for the omnidirectional camera under the constraint of a single view point.

Affine & Scale-Invariant Heterogeneous Pyramid Features for Automatic Matching of High Resolution Pushbroom Imagery from Chang'e 2 Satellite

The automatic feature extracting and matching for large amount of linear pushbroom imagery with higher and higher resolution is urgent and challenging in three dimensional reconstructions, remote sensing and mapping. Affine ＆ scale-invariant heterogeneous pyramid feature is proposed in this paper, along with the new scale-invariant analysis method, the detecting of the key points, the affine ＆ scale-invariant descriptor, the steering method of the matching, and the quasi-dense matching algorithm based on the extensive epipolar geometry. The automatic matching is devised for the linear pushbroom imagery. The whole process is executed on lunar images of the highest resolution of ~7 m/pixel（or ~1 m/pixel in the lower orbits） from the Chinese Chang＇e 2 satellite, it runs robustly at present, and resulting in large amounts of well-distributed-correspondences with accuracy of 0.3 pixels and excellent reliability, which gives great support for the further exploration both on the Moon and the Earth.