Adaptive Recurrent Iterative Updating Stereo Matching Network

When training a stereo matching network with a single training dataset, the network may overly rely on the learned features of the single training dataset due to differences in the training dataset scenes, resulting in poor performance on all datasets. Therefore, feature consistency between matched pixels is a key factor in solving the network’s generalization ability. To address this issue, this paper proposed a more widely applicable stereo matching network that introduced whitening loss into the feature extraction module of stereo matching, and significantly improved the applicability of the network model by constraining the variation between salient feature pixels. In addition, this paper used a GRU iterative update module in the disparity update calculation stage, which expanded the model’s receptive field at multiple resolutions, allowing for precise disparity estimation not only in rich texture areas but also in low texture areas. The model was trained only on the Scene Flow large-scale dataset, and the disparity estimation was conducted on mainstream datasets such as Middlebury, KITTI 2015, and ETH3D. Compared with earlier stereo matching algorithms, this method not only achieves more accurate disparity estimation but also has wider applicability and stronger robustness.

Geometrical-Analysis-Based Algorithm for Stereo Matching of Single-Lens Binocular and Multi-Ocular Stereovision System

A geometrical analysis based algorithm is proposed to achieve the stereo matching of a single-lens prism based stereovision system. By setting the multi- face prism in frontal position of the static CCD （CM-140MCL） camera, equivalent stereo images with different orientations are captured synchronously by virtual cameras which are defined by two boundary lines： the optical axis and CCD camera field of view boundary. Subsequently, the geometrical relationship between the 2D stereo images and corresponding 3D scene is established by employing two fundamentals： ray sketching in which all the pertinent points, lines, and planes are expressed in the 3D camera coordinates and the rule of refraction. Landing on this relationship, the epipolar geometry is thus obtained by fitting a set of corresponding candidate points and thereafter, stereo matching of the prism based stereovision system is obtained. Moreover, the unique geometrical properties of the imaging system allow the proposed method free from the complicated camera calibration procedures and to be easily generalized from binocular and tri-oeular to multi-ocular stereovision systems. The performance of the algorithm is presented through the experiments on the binocular imaging system and the comparison with a conventional projection method demonstrates the efficient assessment of our novel contributions.

Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points （GCPs） is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.

Stereo Matching Method Based on Space-Aware Network Model

The stereo matching method based on a space-aware network is proposed, which divides the network into threesections: Basic layer, scale layer, and decision layer. This division is beneficial to integrate residue network and densenetwork into the space-aware network model. The vertical splitting method for computing matching cost by usingthe space-aware network is proposed for solving the limitation of GPU RAM. Moreover, a hybrid loss is broughtforward to boost the performance of the proposed deep network. In the proposed stereo matching method, thespace-aware network is used to calculate the matching cost and then cross-based cost aggregation and semi-globalmatching are employed to compute a disparity map. Finally, a disparity-post processing method is utilized suchas subpixel interpolation, median filter, and bilateral filter. The experimental results show this method has a goodperformance on running time and accuracy, with a percentage of erroneous pixels of 1.23% on KITTI 2012 and1.94% on KITTI 2015.

Local algorithms on dense two-frame stereo matching

Stereo matching is a fundamental and crucial problem in computer vision. In the last decades, many researchers have been working on it and made great progress. Generally stereo algorithms can be classified into local methods and global methods. In this paper, the challenges of stereo matching are first introduced, and then we focus on local approaches which have simpler structures and higher efficiency than global ones. Local algorithms generally perform four steps： cost computation, cost aggregation, disparity computation and disparity refinement. Every step is deeply investigated, and most work focuses on cost aggregation. We studied most of the classical local methods and divide them into several classes. The classification well illustrates the development history of local stereo correspondence and shows the essence of local matching along with its important and difficult points. At the end we give the future development trend of local methods.

Efficient Stereo Matching Scheme Based on Graph Cuts

A new stereo matching scheme from image pairs based on graph cuts is given,which can solve the problem of large color differences as the result of fusing matching results of graph cuts from different color spaces.This scheme builds normalized histogram and reference histogram from matching results,and uses clustering algorithm to process the two histograms.Region histogram statistical method is adopted to retrieve depth data to achieve final matching results.Regular stereo matching library is used to verify this scheme,and experiments reported in this paper support availability of this method for automatic image processing.This scheme renounces the step of manual selection for adaptive color space and can obtain stable matching results.The whole procedure can be executed automatically and improve the integration level of image analysis process.

An improved seed growth method for accurate stereo matching in disparity space

Matching is a classical problem in stereo vision. To solve the matching problem that components cannot continue growing on the occlusions region and repetitive patterns, an improved seed growth method is proposed. The method obtains a set of interesting points defined as initial seeds from a rectified image. Through global optimization the seeds and their neighbors can be selected in- to a match table. Finally the components grow with the matching points and create a semi-dense map under the maximum similar subset according to the principle of the unique constraint. Experimental results show that the proposed method in the grown process can rectify some errors in matching. The semi-dense map has a good performance in the occlusions region and repetitive patterns. This algorithm is faster and more accurate than the traditional seed growing method.

Fast stereo matching based on edge energy information

A new improvement is proposed for stereo matching which gives a solution to disparity map in terms of edge energy.We decompose the stereo matching into three parts:sparse disparity estimation for image-pairs,edge energy model and final disparity refinement.A three-step procedure is proposed to solve them sequentially.At the first step,we perform an initial disparity model using the ordering constraint and interpolation to obtain a more efficient sparse disparity space.At the second step,we apply the energy function by the edge constraints that exist in both images.The last step is a kind of disparity filling.We determine disparity values in target regions based on global optimization.The proposed three-step simple stereo matching procedure yields excellent quantitative and qualitative results with Middlebury data sets in a fast way.

Fast Stereo Matching Fully Utilizing Super-Pixels

In this paper, we propose a depth image generation method by stereo matching on super-pixel (SP) basis. In the proposed method, block matching is performed only at the center of the SP, and the obtained disparity is applied to all pixels of the SP. Next, in order to improve the disparity, a new SP-based cost filter is introduced. This filter multiplies the matching cost of the surrounding SP by a weight based on reliability and similarity and sums the weighted costs of neighbors. In addition, we propose two new error checking methods. One-way check uses only a unidirectional disparity estimation with a small amount of calculation to detect errors. Cross recovery uses cross checking and error recovery to repair lacks of objects that are problematic with SP-based matching. As a result of the experiment, the execution time of the proposed method using the one-way check was about 1/100 of the full search, and the accuracy was almost equivalent. The accuracy using cross recovery exceeded the full search, and the execution time was about 1/60. Speeding up while maintaining accuracy increases the application range of depth images.

Regular Stereo Matching Improvement System Based on Kinect-supporting Mechanism

In this paper, we built a stereoscopic video associated experimental model, which is referenced as Kinect-supporting improved stereo matching scheme. As the depth maps offered by the Kinect IR-projector are resolution-inadequate, noisy, distance-limited, unstable, and material-sensitive, the appropriated de-noising, stabilization and filtering are first performed for retrieving useful IR-projector depths. The disparities are linearly computed from the refined IR-projector depths to provide specifically referable disparity resources. By exploiting these resources with sufficiency, the proposed mechanism can lead to great enhancement on both speed and accuracy of stereo matching processing to offer better extra virtual view generation and the possibility of price-popularized IR-projector embedded stereoscopic camera.

Stereo Matching of Planar Curves Composed of Time Stamped Points

Matching features such as curve segments in stereo images play a very important role in scene recomtruction. In this paper, a stereo matching algorithm for the trajectories composed of time stamped points is proposed. Based on time stamped points, planar curve match measurements are given first, such as time constraint, cross-ratio invariant constraint and eplpolar geometry constraint; then, a trajectory matching method is proposed based on epipolar geometry constraint and cross-ratio invariant constraint. In order to match the planar curve segments projected by perspective projection system, the curve start time and end time are selected first to prepare match candidates. Then, the epipolar equation is used to discard the unmatched curve segment candidates. At last, a cross ratio invariant constxaint is used to find the most matched curve segments. If their match measurement is higher than the specialized threshold, a candidate with the least cross ratio difference is then selected as the match result; otherwise, no match is found. Unlike the conventional planar curve segments matching algorithm, this paper presents a weakly calibrated binocular stereo vision system which is based on wide baseline. The stamped points are obtained by targets detecting method of flying objects from image sequences. Due to wide baseline, there must exist the projection not in epipolar monotonic order or the curve segments located in very short distance and keeping the epipolar monotonic order. By using the method mentioned above, experiments are made to match planar curve segments not only in epipolar monotonic order but also not in epipolar monotonic order. The results show that the performance of our curve matching algorithm is effective for matching the arc-like planar trajectories composed of time stamped points.

Binocular Vision Object Positioning Method for Robots Based on Coarse-fine Stereo Matching

In order to improve the low positioning accuracy and execution efficiency of the robot binocular vision,a binocular vision positioning method based on coarse-fine stereo matching is proposed to achieve object positioning.The random fern is used in the coarse matching to identify objects in the left and right images,and the pixel coordinates of the object center points in the two images are calculated to complete the center matching.In the fine matching,the right center point is viewed as an estimated value to set the search range of the right image,in which the region matching is implemented to find the best matched point of the left center point.Then,the similar triangle principle of the binocular vision model is used to calculate the 3D coordinates of the center point,achieving fast and accurate object positioning.Finally,the proposed method is applied to the object scene images and the robotic arm grasping platform.The experimental results show that the average absolute positioning error and average relative positioning error of the proposed method are 8.22 mm and 1.96%respectively when the object's depth distance is within 600 mm,the time consumption is less than 1.029s.The method can meet the needs of the robot grasping system,and has better accuracy and robustness.

A stereo matching algorithm using multi-peak candidate matches and geometric constraints

Gray cross correlation matching technique is adopted to extract candidate matches with gray cross correlation coefficients less than some certain range of maximal correlation coefficient called multi-peak candidate matches. Multi-peak candidates are extracted corresponding to three closest feature points at first. The corresponding multi-peak candidate matches are used to construct the model polygon. Correspondence is determined based on the local geometric relations between the three feature points and the multi-peak candidates. The disparity test and the global consistency checkout are applied to eliminate the remaining ambiguous matches that are not removed by the local geometric relational test. Experimental results show that the proposed algorithm is feasible and accurate.

Multilevel Disparity Reconstruction Network for Real-Time Stereo Matching

Recently,stereo matching algorithms based on end-to-end convolutional neural networks achieve excellent performance far exceeding traditional algorithms.Current state-of-the-art stereo matching networks mostly rely on full cost volume and 3D convolutions to regress dense disparity maps.These modules are computationally complex and high consumption of memory,and difficult to deploy in real-time applications.To overcome this problem,we propose multilevel disparity reconstruction network,MDRNet,a lightweight stereo matching network without any 3D convolutions.We use stacked residual pyramids to gradually reconstruct disparity maps from low-level resolution to full-level resolution,replacing common 3D computation and optimization convolutions.Our approach achieves a competitive performance compared with other algorithms on stereo benchmarks and real-time inference at 30 frames per second with 4×104 resolutions.

A Novel Method for Stereo Matching

In this paper we propose a novel edge based, fast and effective stereo matching method, which utilises various geometrical and local area constraints and structural information among edge segments to perform a constraint directed matching. By employing a heuristic labelling technique, the combinatorial search for isomorphic graphs is greatly reduced in the matching process. The implementation and experimental results are presented to show the efficacy of the proposed method.

Joint view synthesis and disparity refinement for stereo matching

Typical stereo algorithms treat disparity estimation and view synthesis as two sequential procedures.In this paper,we consider stereo matching and view synthesis as two complementary components,and present a novel iterative refinement model for joint view synthesis and disparity refinement.To achieve the mutual promotion between view synthesis and disparity refinement,we apply two key strategies,disparity maps fusion and disparity-assisted plane sweep-based rendering(DAPSR).On the one hand,the disparity maps fusion strategy is applied to generate disparity map from synthesized view and input views.This strategy is able to detect and counteract disparity errors caused by potential artifacts from synthesized view.On the other hand,the DAPSR is used for view synthesis and updating,and is able to weaken the interpolation errors caused by outliers in the disparity maps.Experiments on Middlebury benchmarks demonstrate that by introducing the synthesized view,disparity errors due to large occluded region and large baseline are eliminated effectively and the synthesis quality is greatly improved.

Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline

When obtaining three-dimensional(3D)face point cloud data based on structured light,factors related to the environment,occlusion,and illumination intensity lead to holes in the collected data,which affect subsequent recognition.In this study,we propose a hole-filling method based on stereo-matching technology combined with a B-spline.The algorithm uses phase information acquired during raster projection to locate holes in the point cloud,simultaneously extracting boundary point cloud sets.By registering the face point cloud data using the stereo-matching algorithm and the data collected using the raster projection method,some supplementary information points can be obtained at the holes.The shape of the B-spline curve can then be roughly described by a few key points,and the control points are put into the hole area as key points for iterative calculation of surface reconstruction.Simulations using smooth ceramic cups and human face models showed that our model can accurately reproduce details and accurately restore complex shapes on the test surfaces.Simulation results indicated the robustness of the method,which is able to fill holes on complex areas such as the inner side of the nose without a prior model.This approach also effectively supplements the hole information,and the patched point cloud is closer to the original data.This method could be used across a wide range of applications requiring accurate facial recognition.

A stereo matching algorithm based on SIFT feature and homography matrix

Aiming at the low speed of traditional scale-invariant feature transform(SIFT) matching algorithm, an improved matching algorithm is proposed in this paper. Firstly, feature points are detected and the speed of feature points matching is improved by adding epipolar constraint; then according to the matching feature points, the homography matrix is obtained by the least square method; finally, according to the homography matrix, the points in the left image can be mapped into the right image, and if the distance between the mapping point and the matching point in the right image is smaller than the threshold value, the pair of matching points is retained, otherwise discarded. Experimental results show that with the improved matching algorithm, the matching time is reduced by 73.3% and the matching points are entirely correct. In addition, the improved method is robust to rotation and translation.

Real-time stereo matching on CUDA using Fourier descriptors and dynamic programming

Computation of stereoscopic depth and disparity map extraction are dynamic research topics.A large variety of algorithms has been developed,among which we cite feature matching, moment extraction, and image representation using descriptors to determine a disparity map. This paper proposes a new method for stereo matching based on Fourier descriptors. The robustness of these descriptors under photometric and geometric transformations provides a better representation of a template or a local region in the image. In our work, we specifically use generalized Fourier descriptors to compute a robust cost function.Then, a box filter is applied for cost aggregation to enforce a smoothness constraint between neighboring pixels. Optimization and disparity calculation are done using dynamic programming, with a cost based on similarity between generalized Fourier descriptors using Euclidean distance. This local cost function is used to optimize correspondences. Our stereo matching algorithm is evaluated using the Middlebury stereo benchmark; our approach has been implemented on parallel high-performance graphics hardware using CUDA to accelerate our algorithm, giving a real-time implementation.