Adaptive Recurrent Iterative Updating Stereo Matching Network

When training a stereo matching network with a single training dataset, the network may overly rely on the learned features of the single training dataset due to differences in the training dataset scenes, resulting in poor performance on all datasets. Therefore, feature consistency between matched pixels is a key factor in solving the network’s generalization ability. To address this issue, this paper proposed a more widely applicable stereo matching network that introduced whitening loss into the feature extraction module of stereo matching, and significantly improved the applicability of the network model by constraining the variation between salient feature pixels. In addition, this paper used a GRU iterative update module in the disparity update calculation stage, which expanded the model’s receptive field at multiple resolutions, allowing for precise disparity estimation not only in rich texture areas but also in low texture areas. The model was trained only on the Scene Flow large-scale dataset, and the disparity estimation was conducted on mainstream datasets such as Middlebury, KITTI 2015, and ETH3D. Compared with earlier stereo matching algorithms, this method not only achieves more accurate disparity estimation but also has wider applicability and stronger robustness.

Geometrical-Analysis-Based Algorithm for Stereo Matching of Single-Lens Binocular and Multi-Ocular Stereovision System

A geometrical analysis based algorithm is proposed to achieve the stereo matching of a single-lens prism based stereovision system. By setting the multi- face prism in frontal position of the static CCD （CM-140MCL） camera, equivalent stereo images with different orientations are captured synchronously by virtual cameras which are defined by two boundary lines： the optical axis and CCD camera field of view boundary. Subsequently, the geometrical relationship between the 2D stereo images and corresponding 3D scene is established by employing two fundamentals： ray sketching in which all the pertinent points, lines, and planes are expressed in the 3D camera coordinates and the rule of refraction. Landing on this relationship, the epipolar geometry is thus obtained by fitting a set of corresponding candidate points and thereafter, stereo matching of the prism based stereovision system is obtained. Moreover, the unique geometrical properties of the imaging system allow the proposed method free from the complicated camera calibration procedures and to be easily generalized from binocular and tri-oeular to multi-ocular stereovision systems. The performance of the algorithm is presented through the experiments on the binocular imaging system and the comparison with a conventional projection method demonstrates the efficient assessment of our novel contributions.

Stereo Matching Method Based on Space-Aware Network Model

The stereo matching method based on a space-aware network is proposed, which divides the network into threesections: Basic layer, scale layer, and decision layer. This division is beneficial to integrate residue network and densenetwork into the space-aware network model. The vertical splitting method for computing matching cost by usingthe space-aware network is proposed for solving the limitation of GPU RAM. Moreover, a hybrid loss is broughtforward to boost the performance of the proposed deep network. In the proposed stereo matching method, thespace-aware network is used to calculate the matching cost and then cross-based cost aggregation and semi-globalmatching are employed to compute a disparity map. Finally, a disparity-post processing method is utilized suchas subpixel interpolation, median filter, and bilateral filter. The experimental results show this method has a goodperformance on running time and accuracy, with a percentage of erroneous pixels of 1.23% on KITTI 2012 and1.94% on KITTI 2015.

Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points （GCPs） is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.

Local algorithms on dense two-frame stereo matching

Stereo matching is a fundamental and crucial problem in computer vision. In the last decades, many researchers have been working on it and made great progress. Generally stereo algorithms can be classified into local methods and global methods. In this paper, the challenges of stereo matching are first introduced, and then we focus on local approaches which have simpler structures and higher efficiency than global ones. Local algorithms generally perform four steps： cost computation, cost aggregation, disparity computation and disparity refinement. Every step is deeply investigated, and most work focuses on cost aggregation. We studied most of the classical local methods and divide them into several classes. The classification well illustrates the development history of local stereo correspondence and shows the essence of local matching along with its important and difficult points. At the end we give the future development trend of local methods.

Efficient Stereo Matching Scheme Based on Graph Cuts

A new stereo matching scheme from image pairs based on graph cuts is given,which can solve the problem of large color differences as the result of fusing matching results of graph cuts from different color spaces.This scheme builds normalized histogram and reference histogram from matching results,and uses clustering algorithm to process the two histograms.Region histogram statistical method is adopted to retrieve depth data to achieve final matching results.Regular stereo matching library is used to verify this scheme,and experiments reported in this paper support availability of this method for automatic image processing.This scheme renounces the step of manual selection for adaptive color space and can obtain stable matching results.The whole procedure can be executed automatically and improve the integration level of image analysis process.

An improved seed growth method for accurate stereo matching in disparity space

Matching is a classical problem in stereo vision. To solve the matching problem that components cannot continue growing on the occlusions region and repetitive patterns, an improved seed growth method is proposed. The method obtains a set of interesting points defined as initial seeds from a rectified image. Through global optimization the seeds and their neighbors can be selected in- to a match table. Finally the components grow with the matching points and create a semi-dense map under the maximum similar subset according to the principle of the unique constraint. Experimental results show that the proposed method in the grown process can rectify some errors in matching. The semi-dense map has a good performance in the occlusions region and repetitive patterns. This algorithm is faster and more accurate than the traditional seed growing method.

Fast stereo matching based on edge energy information

A new improvement is proposed for stereo matching which gives a solution to disparity map in terms of edge energy.We decompose the stereo matching into three parts:sparse disparity estimation for image-pairs,edge energy model and final disparity refinement.A three-step procedure is proposed to solve them sequentially.At the first step,we perform an initial disparity model using the ordering constraint and interpolation to obtain a more efficient sparse disparity space.At the second step,we apply the energy function by the edge constraints that exist in both images.The last step is a kind of disparity filling.We determine disparity values in target regions based on global optimization.The proposed three-step simple stereo matching procedure yields excellent quantitative and qualitative results with Middlebury data sets in a fast way.

Binocular Vision Object Positioning Method for Robots Based on Coarse-fine Stereo Matching

In order to improve the low positioning accuracy and execution efficiency of the robot binocular vision,a binocular vision positioning method based on coarse-fine stereo matching is proposed to achieve object positioning.The random fern is used in the coarse matching to identify objects in the left and right images,and the pixel coordinates of the object center points in the two images are calculated to complete the center matching.In the fine matching,the right center point is viewed as an estimated value to set the search range of the right image,in which the region matching is implemented to find the best matched point of the left center point.Then,the similar triangle principle of the binocular vision model is used to calculate the 3D coordinates of the center point,achieving fast and accurate object positioning.Finally,the proposed method is applied to the object scene images and the robotic arm grasping platform.The experimental results show that the average absolute positioning error and average relative positioning error of the proposed method are 8.22 mm and 1.96%respectively when the object's depth distance is within 600 mm,the time consumption is less than 1.029s.The method can meet the needs of the robot grasping system,and has better accuracy and robustness.

A stereo matching algorithm using multi-peak candidate matches and geometric constraints

Gray cross correlation matching technique is adopted to extract candidate matches with gray cross correlation coefficients less than some certain range of maximal correlation coefficient called multi-peak candidate matches. Multi-peak candidates are extracted corresponding to three closest feature points at first. The corresponding multi-peak candidate matches are used to construct the model polygon. Correspondence is determined based on the local geometric relations between the three feature points and the multi-peak candidates. The disparity test and the global consistency checkout are applied to eliminate the remaining ambiguous matches that are not removed by the local geometric relational test. Experimental results show that the proposed algorithm is feasible and accurate.

Multilevel Disparity Reconstruction Network for Real-Time Stereo Matching

Recently,stereo matching algorithms based on end-to-end convolutional neural networks achieve excellent performance far exceeding traditional algorithms.Current state-of-the-art stereo matching networks mostly rely on full cost volume and 3D convolutions to regress dense disparity maps.These modules are computationally complex and high consumption of memory,and difficult to deploy in real-time applications.To overcome this problem,we propose multilevel disparity reconstruction network,MDRNet,a lightweight stereo matching network without any 3D convolutions.We use stacked residual pyramids to gradually reconstruct disparity maps from low-level resolution to full-level resolution,replacing common 3D computation and optimization convolutions.Our approach achieves a competitive performance compared with other algorithms on stereo benchmarks and real-time inference at 30 frames per second with 4×104 resolutions.

Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline

When obtaining three-dimensional(3D)face point cloud data based on structured light,factors related to the environment,occlusion,and illumination intensity lead to holes in the collected data,which affect subsequent recognition.In this study,we propose a hole-filling method based on stereo-matching technology combined with a B-spline.The algorithm uses phase information acquired during raster projection to locate holes in the point cloud,simultaneously extracting boundary point cloud sets.By registering the face point cloud data using the stereo-matching algorithm and the data collected using the raster projection method,some supplementary information points can be obtained at the holes.The shape of the B-spline curve can then be roughly described by a few key points,and the control points are put into the hole area as key points for iterative calculation of surface reconstruction.Simulations using smooth ceramic cups and human face models showed that our model can accurately reproduce details and accurately restore complex shapes on the test surfaces.Simulation results indicated the robustness of the method,which is able to fill holes on complex areas such as the inner side of the nose without a prior model.This approach also effectively supplements the hole information,and the patched point cloud is closer to the original data.This method could be used across a wide range of applications requiring accurate facial recognition.

Joint view synthesis and disparity refinement for stereo matching

Typical stereo algorithms treat disparity estimation and view synthesis as two sequential procedures.In this paper,we consider stereo matching and view synthesis as two complementary components,and present a novel iterative refinement model for joint view synthesis and disparity refinement.To achieve the mutual promotion between view synthesis and disparity refinement,we apply two key strategies,disparity maps fusion and disparity-assisted plane sweep-based rendering(DAPSR).On the one hand,the disparity maps fusion strategy is applied to generate disparity map from synthesized view and input views.This strategy is able to detect and counteract disparity errors caused by potential artifacts from synthesized view.On the other hand,the DAPSR is used for view synthesis and updating,and is able to weaken the interpolation errors caused by outliers in the disparity maps.Experiments on Middlebury benchmarks demonstrate that by introducing the synthesized view,disparity errors due to large occluded region and large baseline are eliminated effectively and the synthesis quality is greatly improved.

A Novel Method for Stereo Matching

In this paper we propose a novel edge based, fast and effective stereo matching method, which utilises various geometrical and local area constraints and structural information among edge segments to perform a constraint directed matching. By employing a heuristic labelling technique, the combinatorial search for isomorphic graphs is greatly reduced in the matching process. The implementation and experimental results are presented to show the efficacy of the proposed method.

Feature point matching of curved surface and robust uncertainty

Stereo matching is an important research area in stereovision and stereo matching of curved surface is especially crucial A novel correspondence algorithm is presented and its matching uncertainty is computed robustly for feature points of curved surface. The comers are matched by using homography constraint besides epipolar constraint to solve the occlusion problem. The uncertainty sources are analyzed. A cost function is established and acts as an optimal rule to compute the matching uncertainty. An adaptive scheme Gauss weights are put forward to make the matching results robust to noises. It makes the practical application of comer matching possible. From the experimental results of an image pair of curved surface it is shown that computing uncertainty robustly can restrain the affection caused by noises to the matching precision.

Research on Extraction of Bottom of Shoe Pattern Based on Binocular Stereo Vision

In order to quickly and efficiently get the information of the bottom of the shoe pattern and spraying trajectory, the paper proposes a method based on binocular stereo vision. After acquiring target image, edge detection based on the canny algorithm, the paper begins stereo matching based on area and characteristics of algorithm. To eliminate false matching points, the paper uses the principle of polar geometry in computer vision. For the purpose of gaining the 3D point cloud of spraying curve, the paper adopts the principle of binocular stereo vision 3D measurement, and then carries on cubic spline curve fitting. By HALCON image processing software programming, it proves the feasibility and effectiveness of the method

Improvement of Binocular Reconstruction Algorithm for Measuring 3D Pavement Texture Using a Single Laser Line Scanning Constraint

The dense and accurate measurement of 3D texture is helpful in evaluating the pavement function.To form dense mandatory constraints and improve matching accuracy,the traditional binocular reconstruction technology was improved threefold.First,a single moving laser line was introduced to carry out global scanning constraints on the target,which would well overcome the difficulty of installing and recognizing excessive laser lines.Second,four kinds of improved algorithms,namely,disparity replacement,superposition synthesis,subregion segmentation,and subregion segmentation centroid enhancement,were established based on different constraint mechanism.Last,the improved binocular reconstruction test device was developed to realize the dual functions of 3D texture measurement and precision self-evaluation.Results show that compared with traditional algorithms,the introduction of a single laser line scanning constraint is helpful in improving the measurement’s accuracy.Among various improved algorithms,the improvement effect of the subregion segmentation centroid enhancement method is the best.It has a good effect on both overall measurement and single pointmeasurement,which can be considered to be used in pavement function evaluation.

A deep learning-based binocular perception system

An obstacle perception system for intelligent vehicle is proposed.The proposed system combines the stereo version technique and the deep learning network model,and is applied to obstacle perception tasks in complex environment.In this paper,we provide a complete system design project,which includes the hardware parameters,software framework,algorithm principle,and optimization method.In addition,special experiments are designed to demonstrate that the performance of the proposed system meets the requirements of actual application.The experiment results show that the proposed system is valid to both standard obstacles and non-standard obstacles,and suitable for different weather and lighting conditions in complex environment.It announces that the proposed system is flexible and robust to the intelligent vehicle.

Hybrid tree guided PatchMatch and quantizing acceleration for multiple views disparity estimation

Existing stereo matching methods cannot guarantee both the computational accuracy and efficiency for ihe disparity estimation of large-scale or multi-view images.Hybrid tree method can obtain a disparity estimation fast with relatively low accuracy,while PatchMatch can give high-precision disparity value with relatively high computational cost.In this work,we propose the Hybrid Tree Guided PatchMatch which can calculate the disparity fast and accurate.Firstly,an initial disparity map is estimated by employing hybrid tree cost aggregation,which is used to constrain the label searching range of the PatchMatch.Furthermore,a reliable normal searching range for each current normal vector defined on the initial disparity map is calculated to refine the PatchMatch.Finally,an effective quantizing acceleration strategy is designed to decrease the matching computational cost of continuous disparity.Experimental results demonstrate that the disparity estimation based on our algorithm is better in binocular image benchmarks such as Middlebury and KITTI.We also provide the disparity estimation results for multi-view stereo in real scenes.

Robust statistical approach to stereo disparity maps denoising and refinement

The advent of convolutional neural networks has led to remarkable progress in dense stereo labeling problem,achieving superior performance over the traditional methods.However,the ill-posed nature of stereo matching makes noise(outliers)in winner-takes-all(WTA)disparity maps inevitable.This paper presents a robust statistical approach to noise detection and refinement of WTA disparity maps.In the context of noise detection,the input noisy WTA disparity map is segmented into regular grid cells(regions)with the aim of leveraging Markov random field(MRF)to infer candidate disparity labels.However,there are two key problems:there can be large severe outliers in the regions;second,the regular partition process may produce regions with mixed disparity distributions.To overcome these problems,we optimize a robust objective function over the segmented disparity map.By obtaining the optimal solution of the objective function through a maximum a posteriori estimation in a probabilistic model,we are able to infer MRF candidate disparity labels.We then apply a soft-segmentation constraint on the estimated MRF candidate disparity labels to describe and detect outliers in the disparity map.Next,an edge-preserving statistical inference that leverages the joint statistics of the disparity map and its guidance reference image is used to select correct candidate disparity for each detected outlier.Finally,a weighted median filter is applied to remove small spikes and irregularities in the resulting disparity map.Rigorous and comprehensive experiments showed that the proposed method is distributionally robust and outlier resistant,and can effectively detect and correct outliers in disparity maps.Middlebury evaluation benchmark validated the competitive performance of the proposed method.