An approach to addressing the stereo correspondence problem is presented using genetic algorithms (GAs) to obtain a dense disparity map. Different from previous methods, this approach casts the stereo matching as a mu...An approach to addressing the stereo correspondence problem is presented using genetic algorithms (GAs) to obtain a dense disparity map. Different from previous methods, this approach casts the stereo matching as a multi-extrema optimization problem such that finding the fittest solution from a set of potential disparity maps. Among a wide variety of optimization techniques, GAs are proven to be potentially effective methods for the global optimization problems with large search space. With this idea, each disparity map is viewed as an individual and the disparity values are encoded as chromosomes, so each individual has lots of chromosomes in the approach. Then, several matching constraints are formulated into an objective function, and GAs are used to search the global optimal solution for the problem. Furthermore, the coarse-to-fine strategy has been embedded in the approach so as to reduce the matching ambiguity and the time consumption. Finally, experimental results on synthetic and real images show the performance of the work.展开更多
The perception module of advanced driver assistance systems plays a vital role.Perception schemes often use a single sensor for data processing and environmental perception or adopt the information processing results ...The perception module of advanced driver assistance systems plays a vital role.Perception schemes often use a single sensor for data processing and environmental perception or adopt the information processing results of various sensors for the fusion of the detection layer.This paper proposes a multi-scale and multi-sensor data fusion strategy in the front end of perception and accomplishes a multi-sensor function disparity map generation scheme.A binocular stereo vision sensor composed of two cameras and a light deterction and ranging(LiDAR)sensor is used to jointly perceive the environment,and a multi-scale fusion scheme is employed to improve the accuracy of the disparity map.This solution not only has the advantages of dense perception of binocular stereo vision sensors but also considers the perception accuracy of LiDAR sensors.Experiments demonstrate that the multi-scale multi-sensor scheme proposed in this paper significantly improves disparity map estimation.展开更多
3D reconstruction of environment and weld workpiece can provide geometrical model for telerobotic welding and improve its intelligence. A novel framework of spacetime stereo is employed to overcome the lack of texture...3D reconstruction of environment and weld workpiece can provide geometrical model for telerobotic welding and improve its intelligence. A novel framework of spacetime stereo is employed to overcome the lack of texture of the weld workpiece and obtain subpixel disparity map of the scene. Anisotropic diffusion is adopted to smooth the original subpixel disparity map in order to reduce the noise while preserving the depth discontinuity. A simple algorithm of generation triangle mesh surface from the disparity map of the spucetime stereo is presented. The experimental results of real weld scenes are shown.展开更多
Based on the feature of stereo images' content and the property of natural objects, we redefine the general order matching constraint with object contour restriction. According to the modified order matching const...Based on the feature of stereo images' content and the property of natural objects, we redefine the general order matching constraint with object contour restriction. According to the modified order matching constraint, we propose a robust algorithm for disparity map post processing. Verified by computer simulations using synthetic stereo images with given disparities, our new algorithm proves to be not only efficient in disparity error detection and correction, but also very robust, which can resolve the severe problem in the algorithm proposed in Ref. that if there are large differences among the depths of objects in a scene, the algorithm will make mistakes during the process of disparity error detection and correction.展开更多
The creation of the 3D rendering model involves the prediction of an accurate depth map for the input images.A proposed approach of a modified semi-global block matching algorithm with variable window size and the gra...The creation of the 3D rendering model involves the prediction of an accurate depth map for the input images.A proposed approach of a modified semi-global block matching algorithm with variable window size and the gradient assessment of objects predicts the depth map.3D modeling and view synthesis algorithms could effectively handle the obtained disparity maps.This work uses the consistency check method to find an accurate depth map for identifying occluded pixels.The prediction of the disparity map by semi-global block matching has used the benchmark dataset of Middlebury stereo for evaluation.The improved depth map quality within a reasonable process-ing time outperforms the other existing depth map prediction algorithms.The experimental results have shown that the proposed depth map predictioncould identify the inter-object boundaryeven with the presence ofocclusion with less detection error and runtime.We observed that the Middlebury stereo dataset has very few images with occluded objects,which made the attainment of gain cumbersome.Considering this gain,we have created our dataset with occlu-sion using the structured lighting technique.The proposed regularization term as an optimization process in the graph cut algorithm handles occlusion for different smoothing coefficients.The experimented results demonstrated that our dataset had outperformed the Tsukuba dataset regarding the percentage of occluded pixels.展开更多
Matching is a classical problem in stereo vision. To solve the matching problem that components cannot continue growing on the occlusions region and repetitive patterns, an improved seed growth method is proposed. The...Matching is a classical problem in stereo vision. To solve the matching problem that components cannot continue growing on the occlusions region and repetitive patterns, an improved seed growth method is proposed. The method obtains a set of interesting points defined as initial seeds from a rectified image. Through global optimization the seeds and their neighbors can be selected in- to a match table. Finally the components grow with the matching points and create a semi-dense map under the maximum similar subset according to the principle of the unique constraint. Experimental results show that the proposed method in the grown process can rectify some errors in matching. The semi-dense map has a good performance in the occlusions region and repetitive patterns. This algorithm is faster and more accurate than the traditional seed growing method.展开更多
Distance estimation can be achieved by using active sensors or with the help of passive sensors such as cameras.The stereo vision system is generally composed of two cameras to mimic the human binocular vision.In this...Distance estimation can be achieved by using active sensors or with the help of passive sensors such as cameras.The stereo vision system is generally composed of two cameras to mimic the human binocular vision.In this paper,a Python-based algorithm is pro-posed to find the parameters of each camera,rectify the images,create the disparity maps and finally use these maps for distance measurements.Experiments using real-time im-ages,which were captured from our stereo vision system,of different obstacles posi-tioned at multiple distances(60-200 cm)prove the effectiveness of the proposed program and show that the calculated distance to the obstacle is relatively accurate.The accuracy of distance measurement is up to 99.83%.The processing time needed to calculate the distance between the obstacle and the cameras is less than 0.355 s.展开更多
Road potholes can cause serious social issues,such as unexpected damages to vehicles and traffic accidents.For efficient road management,technologies that quickly find potholes are required,and thus researches on such...Road potholes can cause serious social issues,such as unexpected damages to vehicles and traffic accidents.For efficient road management,technologies that quickly find potholes are required,and thus researches on such technologies have been conducted actively.The three-dimensional(3D)reconstruction method has relatively high accuracy and can be used in practice but it has limited application owing to its long data processing time and high sensor maintenance cost.The two-dimensional(2D)vision method has the advantage of inexpensive and easy application of sensor.Recently,although the 2D vision method using the convolutional neural network(CNN)has shown improved pothole detection performance and adaptability,large amount of data is required to sufficiently train the CNN.Therefore,we propose a method to improve the learning performance of CNN-based object detection model by artificially generating synthetic data similar to a pothole and enhancing the learning data.Additionally,to make the defective areas appear more contrasting,the transformed disparity map(TDM)was calculated using stereo-vision cameras,and the detection performance of the model was further improved through the late fusion with RGB(Red,Green,Blue)images.Consequently,through the convergence of multimodal You Only Look Once(YOLO)frameworks trained by RGB images and TDMs respectively,the detection performance was enhanced by 10.7%compared with that when using only RGB.Further,the superiority of the proposed method was confirmed by showing that the data processing speed was two times faster than the existing 3D reconstruction method.展开更多
文摘An approach to addressing the stereo correspondence problem is presented using genetic algorithms (GAs) to obtain a dense disparity map. Different from previous methods, this approach casts the stereo matching as a multi-extrema optimization problem such that finding the fittest solution from a set of potential disparity maps. Among a wide variety of optimization techniques, GAs are proven to be potentially effective methods for the global optimization problems with large search space. With this idea, each disparity map is viewed as an individual and the disparity values are encoded as chromosomes, so each individual has lots of chromosomes in the approach. Then, several matching constraints are formulated into an objective function, and GAs are used to search the global optimal solution for the problem. Furthermore, the coarse-to-fine strategy has been embedded in the approach so as to reduce the matching ambiguity and the time consumption. Finally, experimental results on synthetic and real images show the performance of the work.
基金the National Key R&D Program of China(2018AAA0103103).
文摘The perception module of advanced driver assistance systems plays a vital role.Perception schemes often use a single sensor for data processing and environmental perception or adopt the information processing results of various sensors for the fusion of the detection layer.This paper proposes a multi-scale and multi-sensor data fusion strategy in the front end of perception and accomplishes a multi-sensor function disparity map generation scheme.A binocular stereo vision sensor composed of two cameras and a light deterction and ranging(LiDAR)sensor is used to jointly perceive the environment,and a multi-scale fusion scheme is employed to improve the accuracy of the disparity map.This solution not only has the advantages of dense perception of binocular stereo vision sensors but also considers the perception accuracy of LiDAR sensors.Experiments demonstrate that the multi-scale multi-sensor scheme proposed in this paper significantly improves disparity map estimation.
文摘3D reconstruction of environment and weld workpiece can provide geometrical model for telerobotic welding and improve its intelligence. A novel framework of spacetime stereo is employed to overcome the lack of texture of the weld workpiece and obtain subpixel disparity map of the scene. Anisotropic diffusion is adopted to smooth the original subpixel disparity map in order to reduce the noise while preserving the depth discontinuity. A simple algorithm of generation triangle mesh surface from the disparity map of the spucetime stereo is presented. The experimental results of real weld scenes are shown.
文摘Based on the feature of stereo images' content and the property of natural objects, we redefine the general order matching constraint with object contour restriction. According to the modified order matching constraint, we propose a robust algorithm for disparity map post processing. Verified by computer simulations using synthetic stereo images with given disparities, our new algorithm proves to be not only efficient in disparity error detection and correction, but also very robust, which can resolve the severe problem in the algorithm proposed in Ref. that if there are large differences among the depths of objects in a scene, the algorithm will make mistakes during the process of disparity error detection and correction.
文摘The creation of the 3D rendering model involves the prediction of an accurate depth map for the input images.A proposed approach of a modified semi-global block matching algorithm with variable window size and the gradient assessment of objects predicts the depth map.3D modeling and view synthesis algorithms could effectively handle the obtained disparity maps.This work uses the consistency check method to find an accurate depth map for identifying occluded pixels.The prediction of the disparity map by semi-global block matching has used the benchmark dataset of Middlebury stereo for evaluation.The improved depth map quality within a reasonable process-ing time outperforms the other existing depth map prediction algorithms.The experimental results have shown that the proposed depth map predictioncould identify the inter-object boundaryeven with the presence ofocclusion with less detection error and runtime.We observed that the Middlebury stereo dataset has very few images with occluded objects,which made the attainment of gain cumbersome.Considering this gain,we have created our dataset with occlu-sion using the structured lighting technique.The proposed regularization term as an optimization process in the graph cut algorithm handles occlusion for different smoothing coefficients.The experimented results demonstrated that our dataset had outperformed the Tsukuba dataset regarding the percentage of occluded pixels.
基金Supported by State Key Laboratory of Explosion Science and Techno logy Foundation(YBKT11-7)
文摘Matching is a classical problem in stereo vision. To solve the matching problem that components cannot continue growing on the occlusions region and repetitive patterns, an improved seed growth method is proposed. The method obtains a set of interesting points defined as initial seeds from a rectified image. Through global optimization the seeds and their neighbors can be selected in- to a match table. Finally the components grow with the matching points and create a semi-dense map under the maximum similar subset according to the principle of the unique constraint. Experimental results show that the proposed method in the grown process can rectify some errors in matching. The semi-dense map has a good performance in the occlusions region and repetitive patterns. This algorithm is faster and more accurate than the traditional seed growing method.
文摘Distance estimation can be achieved by using active sensors or with the help of passive sensors such as cameras.The stereo vision system is generally composed of two cameras to mimic the human binocular vision.In this paper,a Python-based algorithm is pro-posed to find the parameters of each camera,rectify the images,create the disparity maps and finally use these maps for distance measurements.Experiments using real-time im-ages,which were captured from our stereo vision system,of different obstacles posi-tioned at multiple distances(60-200 cm)prove the effectiveness of the proposed program and show that the calculated distance to the obstacle is relatively accurate.The accuracy of distance measurement is up to 99.83%.The processing time needed to calculate the distance between the obstacle and the cameras is less than 0.355 s.
基金This research was funded by a National Research Foundation of Korea(NRF)grant funded by the Korean government(MOE)(No.2021R1I1A3055973)and the Soonchunhyang University Research Fund.
文摘Road potholes can cause serious social issues,such as unexpected damages to vehicles and traffic accidents.For efficient road management,technologies that quickly find potholes are required,and thus researches on such technologies have been conducted actively.The three-dimensional(3D)reconstruction method has relatively high accuracy and can be used in practice but it has limited application owing to its long data processing time and high sensor maintenance cost.The two-dimensional(2D)vision method has the advantage of inexpensive and easy application of sensor.Recently,although the 2D vision method using the convolutional neural network(CNN)has shown improved pothole detection performance and adaptability,large amount of data is required to sufficiently train the CNN.Therefore,we propose a method to improve the learning performance of CNN-based object detection model by artificially generating synthetic data similar to a pothole and enhancing the learning data.Additionally,to make the defective areas appear more contrasting,the transformed disparity map(TDM)was calculated using stereo-vision cameras,and the detection performance of the model was further improved through the late fusion with RGB(Red,Green,Blue)images.Consequently,through the convergence of multimodal You Only Look Once(YOLO)frameworks trained by RGB images and TDMs respectively,the detection performance was enhanced by 10.7%compared with that when using only RGB.Further,the superiority of the proposed method was confirmed by showing that the data processing speed was two times faster than the existing 3D reconstruction method.