Adaptive Recurrent Iterative Updating Stereo Matching Network

When training a stereo matching network with a single training dataset, the network may overly rely on the learned features of the single training dataset due to differences in the training dataset scenes, resulting in poor performance on all datasets. Therefore, feature consistency between matched pixels is a key factor in solving the network’s generalization ability. To address this issue, this paper proposed a more widely applicable stereo matching network that introduced whitening loss into the feature extraction module of stereo matching, and significantly improved the applicability of the network model by constraining the variation between salient feature pixels. In addition, this paper used a GRU iterative update module in the disparity update calculation stage, which expanded the model’s receptive field at multiple resolutions, allowing for precise disparity estimation not only in rich texture areas but also in low texture areas. The model was trained only on the Scene Flow large-scale dataset, and the disparity estimation was conducted on mainstream datasets such as Middlebury, KITTI 2015, and ETH3D. Compared with earlier stereo matching algorithms, this method not only achieves more accurate disparity estimation but also has wider applicability and stronger robustness.

Geometrical-Analysis-Based Algorithm for Stereo Matching of Single-Lens Binocular and Multi-Ocular Stereovision System

A geometrical analysis based algorithm is proposed to achieve the stereo matching of a single-lens prism based stereovision system. By setting the multi- face prism in frontal position of the static CCD （CM-140MCL） camera, equivalent stereo images with different orientations are captured synchronously by virtual cameras which are defined by two boundary lines： the optical axis and CCD camera field of view boundary. Subsequently, the geometrical relationship between the 2D stereo images and corresponding 3D scene is established by employing two fundamentals： ray sketching in which all the pertinent points, lines, and planes are expressed in the 3D camera coordinates and the rule of refraction. Landing on this relationship, the epipolar geometry is thus obtained by fitting a set of corresponding candidate points and thereafter, stereo matching of the prism based stereovision system is obtained. Moreover, the unique geometrical properties of the imaging system allow the proposed method free from the complicated camera calibration procedures and to be easily generalized from binocular and tri-oeular to multi-ocular stereovision systems. The performance of the algorithm is presented through the experiments on the binocular imaging system and the comparison with a conventional projection method demonstrates the efficient assessment of our novel contributions.

Fast Adaptive Support-Weight Stereo Matching Algorithm

Adaptive support-weight (ASW) stereo matching algorithm is widely used in the field of three-dimensional (3D) reconstruction owing to its relatively high matching accuracy.However,since all the weight coefficients need to be calculated in the whole disparity range for each pixel,the algorithm is extremely time-consuming.To solve this problem,a fast ASW algorithm is proposed using twice aggregation.First,a novel weight coefficient which adapts cosine function to satisfy the weight distribution discipline is proposed to accomplish the first cost aggregation.Then,the disparity range is divided into several sub-ranges and local optimal disparities are selected from each of them.For each pixel,only the ASW at the location of local optimal disparities is calculated,and thus,the complexity of the algorithm is greatly reduced.Experimental results show that the proposed algorithm can reduce the amount of calculation by 70% and improve the matching accuracy by 6% for the 15 images on Middlebury Website on average.

Stereo Matching Method Based on Space-Aware Network Model

The stereo matching method based on a space-aware network is proposed, which divides the network into threesections: Basic layer, scale layer, and decision layer. This division is beneficial to integrate residue network and densenetwork into the space-aware network model. The vertical splitting method for computing matching cost by usingthe space-aware network is proposed for solving the limitation of GPU RAM. Moreover, a hybrid loss is broughtforward to boost the performance of the proposed deep network. In the proposed stereo matching method, thespace-aware network is used to calculate the matching cost and then cross-based cost aggregation and semi-globalmatching are employed to compute a disparity map. Finally, a disparity-post processing method is utilized suchas subpixel interpolation, median filter, and bilateral filter. The experimental results show this method has a goodperformance on running time and accuracy, with a percentage of erroneous pixels of 1.23% on KITTI 2012 and1.94% on KITTI 2015.

Local algorithms on dense two-frame stereo matching

Stereo matching is a fundamental and crucial problem in computer vision. In the last decades, many researchers have been working on it and made great progress. Generally stereo algorithms can be classified into local methods and global methods. In this paper, the challenges of stereo matching are first introduced, and then we focus on local approaches which have simpler structures and higher efficiency than global ones. Local algorithms generally perform four steps： cost computation, cost aggregation, disparity computation and disparity refinement. Every step is deeply investigated, and most work focuses on cost aggregation. We studied most of the classical local methods and divide them into several classes. The classification well illustrates the development history of local stereo correspondence and shows the essence of local matching along with its important and difficult points. At the end we give the future development trend of local methods.

Fast Stereo Matching Fully Utilizing Super-Pixels

In this paper, we propose a depth image generation method by stereo matching on super-pixel (SP) basis. In the proposed method, block matching is performed only at the center of the SP, and the obtained disparity is applied to all pixels of the SP. Next, in order to improve the disparity, a new SP-based cost filter is introduced. This filter multiplies the matching cost of the surrounding SP by a weight based on reliability and similarity and sums the weighted costs of neighbors. In addition, we propose two new error checking methods. One-way check uses only a unidirectional disparity estimation with a small amount of calculation to detect errors. Cross recovery uses cross checking and error recovery to repair lacks of objects that are problematic with SP-based matching. As a result of the experiment, the execution time of the proposed method using the one-way check was about 1/100 of the full search, and the accuracy was almost equivalent. The accuracy using cross recovery exceeded the full search, and the execution time was about 1/60. Speeding up while maintaining accuracy increases the application range of depth images.

Regular Stereo Matching Improvement System Based on Kinect-supporting Mechanism

In this paper, we built a stereoscopic video associated experimental model, which is referenced as Kinect-supporting improved stereo matching scheme. As the depth maps offered by the Kinect IR-projector are resolution-inadequate, noisy, distance-limited, unstable, and material-sensitive, the appropriated de-noising, stabilization and filtering are first performed for retrieving useful IR-projector depths. The disparities are linearly computed from the refined IR-projector depths to provide specifically referable disparity resources. By exploiting these resources with sufficiency, the proposed mechanism can lead to great enhancement on both speed and accuracy of stereo matching processing to offer better extra virtual view generation and the possibility of price-popularized IR-projector embedded stereoscopic camera.

基于多特征SAD-Census变换的立体匹配算法

视差不连续区域和重复纹理区域的误匹配率高一直是影响双目立体匹配测量精度的主要问题,为此,本文提出一种基于多特征融合的立体匹配算法。首先,在代价计算阶段,通过高斯加权法赋予邻域像素点的权值,从而优化绝对差之和(Sum of Absolute Differences,SAD)算法的计算精度。接着,基于Census变换改进二进制链码方式,将邻域内像素的平均灰度值与梯度图像的灰度均值相融合,进而建立左右图像对应点的判断依据并优化其编码长度。然后,构建基于十字交叉法与改进的引导滤波器相融合的聚合方法,从而实现视差值再分配,以降低误匹配率。最后,通过赢家通吃(Winner Take All,WTA)算法获取初始视差,并采用左右一致性检测方法及亚像素法提高匹配精度,从而获取最终的视差结果。实验结果表明,在Middlebury数据集的测试中,所提SAD-Census算法的平均非遮挡区域和全部区域的误匹配率为分别为2.67%和5.69%,测量200~900 mm距离的平均误差小于2%;而实际三维测量的最大误差为1.5%。实验结果检验了所提算法的有效性和可靠性。

Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points （GCPs） is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.

Efficient Stereo Matching Scheme Based on Graph Cuts

A new stereo matching scheme from image pairs based on graph cuts is given,which can solve the problem of large color differences as the result of fusing matching results of graph cuts from different color spaces.This scheme builds normalized histogram and reference histogram from matching results,and uses clustering algorithm to process the two histograms.Region histogram statistical method is adopted to retrieve depth data to achieve final matching results.Regular stereo matching library is used to verify this scheme,and experiments reported in this paper support availability of this method for automatic image processing.This scheme renounces the step of manual selection for adaptive color space and can obtain stable matching results.The whole procedure can be executed automatically and improve the integration level of image analysis process.

An improved seed growth method for accurate stereo matching in disparity space

Matching is a classical problem in stereo vision. To solve the matching problem that components cannot continue growing on the occlusions region and repetitive patterns, an improved seed growth method is proposed. The method obtains a set of interesting points defined as initial seeds from a rectified image. Through global optimization the seeds and their neighbors can be selected in- to a match table. Finally the components grow with the matching points and create a semi-dense map under the maximum similar subset according to the principle of the unique constraint. Experimental results show that the proposed method in the grown process can rectify some errors in matching. The semi-dense map has a good performance in the occlusions region and repetitive patterns. This algorithm is faster and more accurate than the traditional seed growing method.

Fast stereo matching based on edge energy information

A new improvement is proposed for stereo matching which gives a solution to disparity map in terms of edge energy.We decompose the stereo matching into three parts:sparse disparity estimation for image-pairs,edge energy model and final disparity refinement.A three-step procedure is proposed to solve them sequentially.At the first step,we perform an initial disparity model using the ordering constraint and interpolation to obtain a more efficient sparse disparity space.At the second step,we apply the energy function by the edge constraints that exist in both images.The last step is a kind of disparity filling.We determine disparity values in target regions based on global optimization.The proposed three-step simple stereo matching procedure yields excellent quantitative and qualitative results with Middlebury data sets in a fast way.

Stereo Matching of Planar Curves Composed of Time Stamped Points

Matching features such as curve segments in stereo images play a very important role in scene recomtruction. In this paper, a stereo matching algorithm for the trajectories composed of time stamped points is proposed. Based on time stamped points, planar curve match measurements are given first, such as time constraint, cross-ratio invariant constraint and eplpolar geometry constraint; then, a trajectory matching method is proposed based on epipolar geometry constraint and cross-ratio invariant constraint. In order to match the planar curve segments projected by perspective projection system, the curve start time and end time are selected first to prepare match candidates. Then, the epipolar equation is used to discard the unmatched curve segment candidates. At last, a cross ratio invariant constxaint is used to find the most matched curve segments. If their match measurement is higher than the specialized threshold, a candidate with the least cross ratio difference is then selected as the match result; otherwise, no match is found. Unlike the conventional planar curve segments matching algorithm, this paper presents a weakly calibrated binocular stereo vision system which is based on wide baseline. The stamped points are obtained by targets detecting method of flying objects from image sequences. Due to wide baseline, there must exist the projection not in epipolar monotonic order or the curve segments located in very short distance and keeping the epipolar monotonic order. By using the method mentioned above, experiments are made to match planar curve segments not only in epipolar monotonic order but also not in epipolar monotonic order. The results show that the performance of our curve matching algorithm is effective for matching the arc-like planar trajectories composed of time stamped points.

结合多路梯度与重排序的AD-Census立体匹配算法

针对局部立体匹配算法在深度不连续和弱纹理区域匹配精度低,且容易受到噪声干扰的问题,提出一种结合多路梯度与重排序的AD-Census立体匹配算法。对传统的AD-Census变换进行改进,重排序不同尺度变换窗口中的像素,取中值作为中心像素点计算Hamming距离,使其对噪声具有更强的鲁棒性,并将八方向梯度信息相融合进行代价计算以增强可靠性;采用十字交叉域和4路径扫描线优化进行代价聚合,提高匹配精度;采用赢家通吃法则进行视差计算,同时引入多步骤视差优化策略得到最终的视差图。在Middlebury测试平台提供的标准立体图像上,所提算法的平均误匹配率低至4.62%,与经典和新颖算法相比,匹配精度明显提高,具有更强的稳健性。

基于联合注意力-可分离卷积的立体匹配算法

为解决立体匹配网络模型轻量化与高精度不能共存的问题,提出新的立体匹配算法CSA-Net(attention and separable convolution network)。该算法是在特征提取阶段,利用类ResNet进行特征提取,训练空洞金字塔池化(atrous spatial pyramid pooling,ASPP)模块扩大感受野,提取多尺度上下文信息,加入联合注意力机制(channel ang spatial attention module,CSM),在空间和通道维度提高表征能力,关注重要特征并抑制不必要的特征。在特征融合阶段,将2D深度可分离卷积提升到3D来代替原网络中标准3D卷积在空间维度和通道维度分别进行卷积运算,以降低特征融合网络的参数量与模型运行时间。实验结果表明,本文所提出的立体匹配网络模型在KITTI 2012和2015数据集进行验证,在三像素匹配误差率为1.44%和2.24%,模型运行时间减少近1/3。因此,相比于其他算法实现了更高的匹配精度和更快的运行速度。

基于截断AD-Census函数的立体匹配算法

双目视觉技术是重要的立体成像技术之一,该技术的核心是计算匹配像素点之间的视差值,而这种计算依赖于代价函数的设计。其中Absolute Difference Census(AD-Census)函数作为代价函数是常用的方法,但该方法受权重系数的影响较大。为了有效提高AD-Census函数的匹配精度,提出向AD-Census函数中添加截断参数以改进其匹配效果,保证了代价聚合时像素的绝对值和梯度不会出现因其中一个值较大,而导致另一个值的影响被忽略的情况出现,从而提高匹配精度,与传统的AD-Census函数相比失配率可以降低6%以上。

Feature point matching of curved surface and robust uncertainty

Stereo matching is an important research area in stereovision and stereo matching of curved surface is especially crucial A novel correspondence algorithm is presented and its matching uncertainty is computed robustly for feature points of curved surface. The comers are matched by using homography constraint besides epipolar constraint to solve the occlusion problem. The uncertainty sources are analyzed. A cost function is established and acts as an optimal rule to compute the matching uncertainty. An adaptive scheme Gauss weights are put forward to make the matching results robust to noises. It makes the practical application of comer matching possible. From the experimental results of an image pair of curved surface it is shown that computing uncertainty robustly can restrain the affection caused by noises to the matching precision.

A Skeletal Camera Network for Close-range Images with a Data Driven Approach in Analyzing Stereo Configuration

Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,the base-height ratio,intersection angle,overlap,and ground control points,etc.,which are rarely quantified in real-world applications.To answer this question,in this paper,we take a data-driven approach by analyzing hundreds of terrestrial stereo image configurations through a typical SfM algorithm.Two main meta-parameters with respect to base-height ratio and intersection angle are analyzed.Following the results,we propose a Skeletal Camera Network(SCN)and embed it into the SfM to lead to a novel SfM scheme called SCN-SfM,which limits tie-point matching to the remaining connected image pairs in SCN.The proposed method was applied in three terrestrial datasets.Experimental results have demonstrated the effectiveness of the proposed SCN-SfM to achieve 3D geometry with higher accuracy and fast time efficiency compared to the typical SfM method,whereas the completeness of the geometry is comparable.