REAL-TIME STEREO MATCHING ALGORITHM IN PHOTODYNAMIC THERAPY BINOCULAR SURVEILLANCE SYSTEM FOR PORT WINE STAIN

A stereo matching algorithm based on the epipolar line constraint is designed to meet the real-time and the accuracy requirements. The algorithm is applied to photodynamic therapy binocular surveillance system for port wine stain （PWS） when it monitors the position of the treatment region. The corner matching based on Hu moments is used to calculate the fundamental matrix of the binocular vision system. Experimental results are in agreement with the theoretical calculation.

Geometrical-Analysis-Based Algorithm for Stereo Matching of Single-Lens Binocular and Multi-Ocular Stereovision System

A geometrical analysis based algorithm is proposed to achieve the stereo matching of a single-lens prism based stereovision system. By setting the multi- face prism in frontal position of the static CCD （CM-140MCL） camera, equivalent stereo images with different orientations are captured synchronously by virtual cameras which are defined by two boundary lines： the optical axis and CCD camera field of view boundary. Subsequently, the geometrical relationship between the 2D stereo images and corresponding 3D scene is established by employing two fundamentals： ray sketching in which all the pertinent points, lines, and planes are expressed in the 3D camera coordinates and the rule of refraction. Landing on this relationship, the epipolar geometry is thus obtained by fitting a set of corresponding candidate points and thereafter, stereo matching of the prism based stereovision system is obtained. Moreover, the unique geometrical properties of the imaging system allow the proposed method free from the complicated camera calibration procedures and to be easily generalized from binocular and tri-oeular to multi-ocular stereovision systems. The performance of the algorithm is presented through the experiments on the binocular imaging system and the comparison with a conventional projection method demonstrates the efficient assessment of our novel contributions.

Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points （GCPs） is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.

Stereo Matching Method Based on Space-Aware Network Model

The stereo matching method based on a space-aware network is proposed, which divides the network into threesections: Basic layer, scale layer, and decision layer. This division is beneficial to integrate residue network and densenetwork into the space-aware network model. The vertical splitting method for computing matching cost by usingthe space-aware network is proposed for solving the limitation of GPU RAM. Moreover, a hybrid loss is broughtforward to boost the performance of the proposed deep network. In the proposed stereo matching method, thespace-aware network is used to calculate the matching cost and then cross-based cost aggregation and semi-globalmatching are employed to compute a disparity map. Finally, a disparity-post processing method is utilized suchas subpixel interpolation, median filter, and bilateral filter. The experimental results show this method has a goodperformance on running time and accuracy, with a percentage of erroneous pixels of 1.23% on KITTI 2012 and1.94% on KITTI 2015.

Fast Adaptive Support-Weight Stereo Matching Algorithm

Adaptive support-weight (ASW) stereo matching algorithm is widely used in the field of three-dimensional (3D) reconstruction owing to its relatively high matching accuracy. However, since all the weight coefficients need to be calculated in the whole disparity range for each pixel, the algorithm is extremely time-consuming. To solve this problem, a fast ASW algorithm is proposed using twice aggregation. First, a novel weight coefficient which adapts cosine function to satisfy the weight distribution discipline is proposed to accomplish the first cost aggregation. Then, the disparity range is divided into several sub-ranges and local optimal disparities are selected from each of them. For each pixel, only the ASW at the location of local optimal disparities is calculated, and thus, the complexity of the algorithm is greatly reduced. Experimental results show that the proposed algorithm can reduce the amount of calculation by 70% and improve the matching accuracy by 6% for the 15 images on Middlebury Website on average. © 2017, Tianjin University and Springer-Verlag Berlin Heidelberg.

Local algorithms on dense two-frame stereo matching

Stereo matching is a fundamental and crucial problem in computer vision. In the last decades, many researchers have been working on it and made great progress. Generally stereo algorithms can be classified into local methods and global methods. In this paper, the challenges of stereo matching are first introduced, and then we focus on local approaches which have simpler structures and higher efficiency than global ones. Local algorithms generally perform four steps： cost computation, cost aggregation, disparity computation and disparity refinement. Every step is deeply investigated, and most work focuses on cost aggregation. We studied most of the classical local methods and divide them into several classes. The classification well illustrates the development history of local stereo correspondence and shows the essence of local matching along with its important and difficult points. At the end we give the future development trend of local methods.

Efficient Stereo Matching Scheme Based on Graph Cuts

A new stereo matching scheme from image pairs based on graph cuts is given,which can solve the problem of large color differences as the result of fusing matching results of graph cuts from different color spaces.This scheme builds normalized histogram and reference histogram from matching results,and uses clustering algorithm to process the two histograms.Region histogram statistical method is adopted to retrieve depth data to achieve final matching results.Regular stereo matching library is used to verify this scheme,and experiments reported in this paper support availability of this method for automatic image processing.This scheme renounces the step of manual selection for adaptive color space and can obtain stable matching results.The whole procedure can be executed automatically and improve the integration level of image analysis process.

An improved seed growth method for accurate stereo matching in disparity space

Matching is a classical problem in stereo vision. To solve the matching problem that components cannot continue growing on the occlusions region and repetitive patterns, an improved seed growth method is proposed. The method obtains a set of interesting points defined as initial seeds from a rectified image. Through global optimization the seeds and their neighbors can be selected in- to a match table. Finally the components grow with the matching points and create a semi-dense map under the maximum similar subset according to the principle of the unique constraint. Experimental results show that the proposed method in the grown process can rectify some errors in matching. The semi-dense map has a good performance in the occlusions region and repetitive patterns. This algorithm is faster and more accurate than the traditional seed growing method.

Fast stereo matching based on edge energy information

A new improvement is proposed for stereo matching which gives a solution to disparity map in terms of edge energy.We decompose the stereo matching into three parts:sparse disparity estimation for image-pairs,edge energy model and final disparity refinement.A three-step procedure is proposed to solve them sequentially.At the first step,we perform an initial disparity model using the ordering constraint and interpolation to obtain a more efficient sparse disparity space.At the second step,we apply the energy function by the edge constraints that exist in both images.The last step is a kind of disparity filling.We determine disparity values in target regions based on global optimization.The proposed three-step simple stereo matching procedure yields excellent quantitative and qualitative results with Middlebury data sets in a fast way.

Stereo Matching of Planar Curves Composed of Time Stamped Points

Matching features such as curve segments in stereo images play a very important role in scene recomtruction. In this paper, a stereo matching algorithm for the trajectories composed of time stamped points is proposed. Based on time stamped points, planar curve match measurements are given first, such as time constraint, cross-ratio invariant constraint and eplpolar geometry constraint; then, a trajectory matching method is proposed based on epipolar geometry constraint and cross-ratio invariant constraint. In order to match the planar curve segments projected by perspective projection system, the curve start time and end time are selected first to prepare match candidates. Then, the epipolar equation is used to discard the unmatched curve segment candidates. At last, a cross ratio invariant constxaint is used to find the most matched curve segments. If their match measurement is higher than the specialized threshold, a candidate with the least cross ratio difference is then selected as the match result; otherwise, no match is found. Unlike the conventional planar curve segments matching algorithm, this paper presents a weakly calibrated binocular stereo vision system which is based on wide baseline. The stamped points are obtained by targets detecting method of flying objects from image sequences. Due to wide baseline, there must exist the projection not in epipolar monotonic order or the curve segments located in very short distance and keeping the epipolar monotonic order. By using the method mentioned above, experiments are made to match planar curve segments not only in epipolar monotonic order but also not in epipolar monotonic order. The results show that the performance of our curve matching algorithm is effective for matching the arc-like planar trajectories composed of time stamped points.

Fast Stereo Matching Fully Utilizing Super-Pixels

In this paper, we propose a depth image generation method by stereo matching on super-pixel (SP) basis. In the proposed method, block matching is performed only at the center of the SP, and the obtained disparity is applied to all pixels of the SP. Next, in order to improve the disparity, a new SP-based cost filter is introduced. This filter multiplies the matching cost of the surrounding SP by a weight based on reliability and similarity and sums the weighted costs of neighbors. In addition, we propose two new error checking methods. One-way check uses only a unidirectional disparity estimation with a small amount of calculation to detect errors. Cross recovery uses cross checking and error recovery to repair lacks of objects that are problematic with SP-based matching. As a result of the experiment, the execution time of the proposed method using the one-way check was about 1/100 of the full search, and the accuracy was almost equivalent. The accuracy using cross recovery exceeded the full search, and the execution time was about 1/60. Speeding up while maintaining accuracy increases the application range of depth images.

Regular Stereo Matching Improvement System Based on Kinect-supporting Mechanism

In this paper, we built a stereoscopic video associated experimental model, which is referenced as Kinect-supporting improved stereo matching scheme. As the depth maps offered by the Kinect IR-projector are resolution-inadequate, noisy, distance-limited, unstable, and material-sensitive, the appropriated de-noising, stabilization and filtering are first performed for retrieving useful IR-projector depths. The disparities are linearly computed from the refined IR-projector depths to provide specifically referable disparity resources. By exploiting these resources with sufficiency, the proposed mechanism can lead to great enhancement on both speed and accuracy of stereo matching processing to offer better extra virtual view generation and the possibility of price-popularized IR-projector embedded stereoscopic camera.

Adaptive Recurrent Iterative Updating Stereo Matching Network

When training a stereo matching network with a single training dataset, the network may overly rely on the learned features of the single training dataset due to differences in the training dataset scenes, resulting in poor performance on all datasets. Therefore, feature consistency between matched pixels is a key factor in solving the network’s generalization ability. To address this issue, this paper proposed a more widely applicable stereo matching network that introduced whitening loss into the feature extraction module of stereo matching, and significantly improved the applicability of the network model by constraining the variation between salient feature pixels. In addition, this paper used a GRU iterative update module in the disparity update calculation stage, which expanded the model’s receptive field at multiple resolutions, allowing for precise disparity estimation not only in rich texture areas but also in low texture areas. The model was trained only on the Scene Flow large-scale dataset, and the disparity estimation was conducted on mainstream datasets such as Middlebury, KITTI 2015, and ETH3D. Compared with earlier stereo matching algorithms, this method not only achieves more accurate disparity estimation but also has wider applicability and stronger robustness.

Feature point matching of curved surface and robust uncertainty

Stereo matching is an important research area in stereovision and stereo matching of curved surface is especially crucial A novel correspondence algorithm is presented and its matching uncertainty is computed robustly for feature points of curved surface. The comers are matched by using homography constraint besides epipolar constraint to solve the occlusion problem. The uncertainty sources are analyzed. A cost function is established and acts as an optimal rule to compute the matching uncertainty. An adaptive scheme Gauss weights are put forward to make the matching results robust to noises. It makes the practical application of comer matching possible. From the experimental results of an image pair of curved surface it is shown that computing uncertainty robustly can restrain the affection caused by noises to the matching precision.

基于双目视觉技术的茶叶嫩芽定位方法研究

进一步提升茶叶嫩芽定位的准确性及茶叶嫩芽采摘的作业效率,选取双目视觉技术作为切入点,针对其识别、定位、采摘等环节展开设计研究。考虑茶叶嫩芽采摘的基础特征和双目视觉技术的应用条件,合理搭建双目视觉模型,选用立体匹配的核心定位算法,完成茶叶嫩芽的定位方法与采摘功能优化,并布置相应的试验机构,展开定位采摘验证试验。结果表明:基于双目视觉技术的茶叶嫩芽定位方法应用具有显著的优越性,较普通定位方法而言,识别误匹配率可降低至4.01%,平均漏采率为3.20%,嫩芽采摘完整率达93.40%,可以很好地促进茶叶嫩芽的后续处理加工,是采摘茶叶嫩芽的品质化的重要基础保障。

Wheel center detection based on stereo vision

As the location of the wheel center is the key to accurately measuring the wheelbase, the wheelbase difference and the wheel static radius, a high-precision wheel center detection method based on stereo vision is proposed. First, according to the prior information, the contour of the wheel hub is extracted and fitted as an ellipse curve, and the ellipse fitting equation can be obtained. Then, a new un-tangent constraint is adopted to improve the ellipse matching precision. Finally, the 3D coordinates of the wheel center can be reconstructed by the spatial circle projection algorithm with low time complexity and high measurement accuracy. Simulation experiments verify that compared with the ellipse center reconstruction algorithm and the planar constraint optimization algorithm, the proposed method can acquire the 3D coordinates of the spatial circle more exactly. Furthermore, the measurements of the wheelbase, the wheelbase difference and the wheel static radius for three types of vehicles demonstrate the effectiveness of the proposed method for wheel center detection.

基于改进Patchmatch及切片采样粒子置信度传播的立体匹配算法

针对立体匹配时视差不连续区、倾斜平面及非前向平行平面误匹配较高的问题,提出了一种基于改进Patchmatch及切片采样粒子置信度的立体匹配算法.定义了具有边缘特性的Patchmatch相似性函数,并建立基于Patchmatch的非前向平行平面视差平面估计模型.利用粒子置信度传播代替原有的最近邻搜索,使用较少的粒子近似目标分布,并采用切片采样马尔可夫链蒙特卡罗方法解决传播过程中粒子重采样更新问题.Middlebury图像数据集测试表明,该算法能够降低视差不连续区域的误匹配,有效地提高了倾斜平面及非前向平行平面图像的匹配精度.

基于跨尺度PatchMatch的立体匹配算法

针对现有的PatchMatch(3D标签优化)立体匹配算法存在对图像中弱纹理、视差不连续区域匹配精度低的问题,提出了一种结合超像素分割和跨尺度PatchMatch的立体匹配算法。首先,通过高斯下采样获得多尺度图像并对各尺度图像超像素分割。其次,基于四色定理腐蚀超像素边界使3D标签在超像素上迭代传播具有子模性和独立性,生成的子模能量用图割(graph cut, GC)算法得到最优解。最后,提出跨尺度能量函数模型,约束不同尺度下同名像素3D标签能量一致,使3D标签迭代传播可在不同尺度进行GC优化,获得最优视差图。在Middlebury数据集上的实验结果表明,本文算法对21组弱纹理、复杂纹理图像的平均误匹配率为2.20%,相比其他改进的PatchMatch立体匹配算法误匹配率降低了10.1%,且视差图误匹配可视化显示,弱纹理、视差不连续区域匹配效果优于其他改进的PatchMatch立体匹配算法。