Monocular Depth Estimation with Sharp Boundary

Monocular depth estimation is the basic task in computer vision.Its accuracy has tremendous improvement in the decade with the development of deep learning.However,the blurry boundary in the depth map is a serious problem.Researchers find that the blurry boundary is mainly caused by two factors.First,the low-level features,containing boundary and structure information,may be lost in deep networks during the convolution process.Second,themodel ignores the errors introduced by the boundary area due to the few portions of the boundary area in the whole area,during the backpropagation.Focusing on the factors mentioned above.Two countermeasures are proposed to mitigate the boundary blur problem.Firstly,we design a scene understanding module and scale transformmodule to build a lightweight fuse feature pyramid,which can deal with low-level feature loss effectively.Secondly,we propose a boundary-aware depth loss function to pay attention to the effects of the boundary’s depth value.Extensive experiments show that our method can predict the depth maps with clearer boundaries,and the performance of the depth accuracy based on NYU-Depth V2,SUN RGB-D,and iBims-1 are competitive.

Improved block matching approach to fast disparity estimation

An improved block matching approach to fast disparity estimation in machine vision applications is proposed, where the matching criterion is the sum of the absolute difference（SAD）.By evaluating the lower bounds, which become increasingly tighter for the matching criteria, the method tries to successively terminate unnecessary computations of the matching criteria between the reference block in one image and the ineligible candidate blocks in another image.It also eliminates the ineligible blocks as early as possible, while ensuring the optimal disparity of each pixel.Also, the proposed method can further speed up the elimination of ineligible candidate blocks by efficiently using the continuous constraint of disparity to predict the initial disparity of each pixel.The performance of the new algorithm is evaluated by carrying out a theoretical analysis, and by comparing its performance with the disparity estimation method based on the standard block matching.Simulated results demonstrate that the proposed algorithm achieves a computational cost reduction of over 50.5% in comparision with the standard block matching method.

Hybrid tree guided PatchMatch and quantizing acceleration for multiple views disparity estimation

Existing stereo matching methods cannot guarantee both the computational accuracy and efficiency for ihe disparity estimation of large-scale or multi-view images.Hybrid tree method can obtain a disparity estimation fast with relatively low accuracy,while PatchMatch can give high-precision disparity value with relatively high computational cost.In this work,we propose the Hybrid Tree Guided PatchMatch which can calculate the disparity fast and accurate.Firstly,an initial disparity map is estimated by employing hybrid tree cost aggregation,which is used to constrain the label searching range of the PatchMatch.Furthermore,a reliable normal searching range for each current normal vector defined on the initial disparity map is calculated to refine the PatchMatch.Finally,an effective quantizing acceleration strategy is designed to decrease the matching computational cost of continuous disparity.Experimental results demonstrate that the disparity estimation based on our algorithm is better in binocular image benchmarks such as Middlebury and KITTI.We also provide the disparity estimation results for multi-view stereo in real scenes.

基于改进FeatDepth的足球运动场景无监督单目图像深度预测

为了在降低成本的同时提高图像深度信息预测的精确度,并将深度估计应用于足球运动场景,提出一种基于改进FeatDepth的足球运动场景无监督单目图像深度预测方法。首先,对原FeatDepth引入注意力机制,使模型更加关注有效的特征信息;其次,将FeatDepth中的PoseNet网络和DepthNet网络分别嵌入GAM全局注意力机制模块,为网络添加额外的上下文信息,在基本不增加计算成本的情况下提升FeatDepth模型深度预测性能;再次,为在低纹理区域和细节上获得更好的深度预测效果,由单视图重构损失与交叉视图重构损失组合而成最终的损失函数。选取KITTI数据集中Person场景较多的部分进行数据集制作并进行仿真实验,结果表明,改进后的FeatDepth模型不仅在精确度上有所提升,且在低纹理区域及细节处拥有更好的深度预测效果。最后,对比模型在足球场景下的推理效果后得出,改进后的模型在低纹理区域(足球、球门等)及细节处(肢体等)有更好的深度预测效果,实现了将基于无监督的单目深度估计模型应用于足球运动场景的目的。

Geophysical Study: Estimation of Deposit Depth Using Gravimetric Data and Euler Method (Jalalabad Iron Mine, Kerman Province of IRAN)

Mineral exploration is done by different methods. Geophysical and geochemical studies are two powerful tools in this field. In integrated studies, the results of each study are used to determine the location of the drilling boreholes. The purpose of this study is to use field geophysics to calculate the depth of mineral reserve. The study area is located 38 km from Zarand city called Jalalabad iron mine. In this study, gravimetric data were measured and mineral depth was calculated using the Euler method. 1314 readings have been performed in this area. The rocks of the region include volcanic and sedimentary. The source of the mineralization in the area is hydrothermal processes. After gravity measuring in the region, the data were corrected, then various methods such as anomalous map remaining in levels one and two, upward expansion, first and second-degree vertical derivatives, analytical method, and analytical signal were drawn, and finally, the depth of the deposit was estimated by Euler method. As a result, the depth of the mineral deposit was calculated to be between 20 and 30 meters on average.

A Simple Method for Source Depth Estimation with Multi-path Time Delay in Deep Ocean

A method of source depth estimation based on the multi-path time delay difference is proposed. When the minimum time arrivals in all receiver depths are snapped to a certain time on time delay-depth plane, time delay arrivals of surface-bottom reflection and bottom-surface reflection intersect at the source depth. Two hydrophones deployed vertically with a certain interval are required at least. If the receiver depths are known, the pair of time delays can be used to estimate the source depth. With the proposed method the source depth can be estimated successfully in a moderate range in the deep ocean without complicated matched-field calculations in the simulations and experiments.

Temporally Consistent Depth Map Estimation for 3D Video Generation and Coding

In this paper, we propose a new algorithm for temporally consistent depth map estimation to generate three-dimensional video. The proposed algorithm adaptively computes the matching cost using a temporal weighting function, which is obtained by block-based moving object detection and motion estimation with variable block sizes. Experimental results show that the proposed algorithm improves the temporal consistency of the depth video and reduces by about 38% both the flickering artefact in the synthesized view and the number of coding bits for depth video coding.

基于Shuffle-ZoeDepth单目深度估计的苗期玉米株高测量方法

株高是鉴别玉米种质性状及作物活力的重要表型指标,苗期玉米遗传特性表现明显,准确测量苗期玉米植株高度对玉米遗传特性鉴别与田间管理具有重要意义。针对传统植株高度获取方法依赖人工测量,费时费力且存在主观误差的问题,提出了一种融合混合注意力信息的改进ZoeDepth单目深度估计模型。改进后的模型将Shuffle Attention模块加入Decoder模块的4个阶段,使Decoder模块在对低分辨率特征图信息提取过程中能更关注特征图中的有效信息,提升了模型关键信息的提取能力,可生成更精确的深度图。为验证本研究方法的有效性,在NYU-V2深度数据集上进行了验证。结果表明,改进的Shuffle-ZoeDepth模型在NYU-V2深度数据集上绝对相对差、均方根误差、对数均方根误差为0.083、0.301 mm、0.036,不同阈值下准确率分别为93.9%、99.1%、99.8%,均优于ZoeDepth模型。同时,利用Shuffle-ZoeDepth单目深度估计模型结合玉米植株高度测量模型实现了苗期玉米植株高度的测量,采集不同距离下苗期玉米图像进行植株高度测量试验。当玉米高度在15~25 cm、25~35 cm、35~45 cm 3个区间时,平均测量绝对误差分别为1.41、2.21、2.08 cm,平均测量百分比误差分别为8.41%、7.54%、4.98%。试验结果表明该方法可仅使用单个RGB相机完成复杂室外环境下苗期玉米植株高度的精确测量。

Perpendicular-Cutdepth:Perpendicular Direction Depth Cutting Data Augmentation Method

Depth estimation is an important task in computer vision.Collecting data at scale for monocular depth estimation is challenging,as this task requires simultaneously capturing RGB images and depth information.Therefore,data augmentation is crucial for this task.Existing data augmentationmethods often employ pixel-wise transformations,whichmay inadvertently disrupt edge features.In this paper,we propose a data augmentationmethod formonocular depth estimation,which we refer to as the Perpendicular-Cutdepth method.This method involves cutting realworld depth maps along perpendicular directions and pasting them onto input images,thereby diversifying the data without compromising edge features.To validate the effectiveness of the algorithm,we compared it with existing convolutional neural network(CNN)against the current mainstream data augmentation algorithms.Additionally,to verify the algorithm’s applicability to Transformer networks,we designed an encoder-decoder network structure based on Transformer to assess the generalization of our proposed algorithm.Experimental results demonstrate that,in the field of monocular depth estimation,our proposed method,Perpendicular-Cutdepth,outperforms traditional data augmentationmethods.On the indoor dataset NYU,our method increases accuracy from0.900 to 0.907 and reduces the error rate from0.357 to 0.351.On the outdoor dataset KITTI,our method improves accuracy from 0.9638 to 0.9642 and decreases the error rate from 0.060 to 0.0598.

Depth estimation system suitable for hardware design

Depth estimation is an active research area with the developing of stereo vision in recent years. It is one of the key technologies to resolve the large data of stereo vision communication. Now depth estimation still has some problems, such as occlusion, fuzzy edge, real-time processing, etc. Many algorithms have been proposed base on software, however the performance of the computer configurations limits the software processing speed. The other resolution is hardware design and the great developments of the digital signal processor （DSP）, and application specific integrated circuit （ASIC） and field programmable gate array （FPGA） provide the opportunity of flexible applications. In this work, by analyzing the procedures of depth estimation, the proper algorithms which can be used in hardware design to execute real-time depth estimation are proposed. The different methods of calibration, matching and post-processing are analyzed based on the hardware design requirements. At last some tests for the algorithm have been analyzed. The results show that the algorithms proposed for hardware design can provide credited depth map for further view synthesis and are suitable for hardware design.

Fault depth estimation using support vector classifier and features selection

Depth estimation of subsurface faults is one of the problems in gravity interpretation. We tried using the support vector classifier （SVC） method in the estimation. Using forward and nonlinear inverse techniques, detecting the depth of subsurface faults with related error is possible but it is necessary to have an initial guess for the depth and this initial guess usually comes from non-gravity data. We introduce SVC in this paper as one of the tools for estimating the depth of subsurface faults using gravity data. We can suppose that each subsurface fault depth is a class and that SVC is a classification algorithm. To better use the SVC algorithm, we select proper depth estimation features using a proper features selection （FS） algorithm. In this research, we produce a training set consisting of synthetic gravity profiles created by subsurface faults at different depths to train the SVC code to estimate the depth of real subsurface faults. Then we test our trained SVC code by a testing set consisting of other synthetic gravity profiles created by subsurface faults at different depths. We also tested our trained SVC code using real data.

Depth Estimation from a Single Image Based on Cauchy Distribution Model

Most approaches to estimate a scene’s 3D depth from a single image often model the point spread function (PSF) as a 2D Gaussian function. However, those methods are suffered from some noises, and difficult to get a high quality of depth recovery. We presented a simple yet effective approach to estimate exactly the amount of spatially varying defocus blur at edges, based on a Cauchy distribution model for the PSF. The raw image was re-blurred twice using two known Cauchy distribution kernels, and the defocus blur amount at edges could be derived from the gradient ratio between the two re-blurred images. By propagating the blur amount at edge locations to the entire image using the matting interpolation, a full depth map was then recovered. Experimental results on several real images demonstrated both feasibility and effectiveness of our method, being a non-Gaussian model for DSF, in providing a better estimation of the defocus map from a single un-calibrated defocused image. These results also showed that our method was robust to image noises, inaccurate edge location and interferences of neighboring edges. It could generate more accurate scene depth maps than the most of existing methods using a Gaussian based DSF model.

Boosting Unsupervised Monocular Depth Estimation with Auxiliary Semantic Information

Learning-based multi-task models have been widely used in various scene understanding tasks,and complement each other,i.e.,they allow us to consider prior semantic information to better infer depth.We boost the unsupervised monocular depth estimation using semantic segmentation as an auxiliary task.To address the lack of cross-domain datasets and catastrophic forgetting problems encountered in multi-task training,we utilize existing methodology to obtain redundant segmentation maps to build our cross-domain dataset,which not only provides a new way to conduct multi-task training,but also helps us to evaluate results compared with those of other algorithms.In addition,in order to comprehensively use the extracted features of the two tasks in the early perception stage,we use a strategy of sharing weights in the network to fuse cross-domain features,and introduce a novel multi-task loss function to further smooth the depth values.Extensive experiments on KITTI and Cityscapes datasets show that our method has achieved state-of-the-art performance in the depth estimation task,as well improved semantic segmentation.

A method to generate foggy optical images based on unsupervised depth estimation

For traffic object detection in foggy environment based on convolutional neural network(CNN),data sets in fog-free environment are generally used to train the network directly.As a result,the network cannot learn the object characteristics in the foggy environment in the training set,and the detection effect is not good.To improve the traffic object detection in foggy environment,we propose a method of generating foggy images on fog-free images from the perspective of data set construction.First,taking the KITTI objection detection data set as an original fog-free image,we generate the depth image of the original image by using improved Monodepth unsupervised depth estimation method.Then,a geometric prior depth template is constructed to fuse the image entropy taken as weight with the depth image.After that,a foggy image is acquired from the depth image based on the atmospheric scattering model.Finally,we take two typical object-detection frameworks,that is,the two-stage object-detection Fster region-based convolutional neural network(Faster-RCNN)and the one-stage object-detection network YOLOv4,to train the original data set,the foggy data set and the mixed data set,respectively.According to the test results on RESIDE-RTTS data set in the outdoor natural foggy environment,the model under the training on the mixed data set shows the best effect.The mean average precision(mAP)values are increased by 5.6%and by 5.0%under the YOLOv4 model and the Faster-RCNN network,respectively.It is proved that the proposed method can effectively improve object identification ability foggy environment.

RADepthNet:Reflectance-aware monocular depth estimation

Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy,which leads to inferior performance.Methods To remove the influence of depth-irrelevant information and improve the depth-prediction accuracy,we propose RADepthNet,a novel reflectance-guided network that fuses boundary features.Specifically,our method predicts depth maps using the following three steps:(1)Intrinsic Image Decomposition.We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance.Through an ablation study,we demonstrate that the module can reduce the influence of illumination on depth estimation.(2)Boundary Detection.A boundary extraction module,consisting of an encoder,refinement block,and upsample block,was proposed to better predict the depth at object boundaries utilizing gradient constraints.(3)Depth Prediction Module.We use an encoder different from(2)to obtain depth features from the reflectance map and fuse boundary features to predict depth.In addition,we proposed FIFADataset,a depth-estimation dataset applied in soccer scenarios.Results Extensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.

基于Boosting-Monodepth的管道病害深度估计与三维重建

城市地下管道是城市的血脉经络,但随着排水管道的大量投入运营和使用年限增加,引发了一系列的管道病害安全隐患,如管道整体结构变形、内表面破裂和管中异物插入等问题,传统的病害图像视频采集、检测和后期病害分类甄选都是从二维视角出发,欠缺对三维空间信息(深度)的考虑。针对上述3种病害从生成深度图、由二维深度图重建三维管道病害这两方面进行研究,提出了一种基于boosting-monodepth的双重深度估计方法以提升深度图效果,最终生成画面连续一致、轮廓清晰的深度图。性能评估方面采用Abs-Rel、RMSE、SqRel、ORD和D3R等通用指标,与传统算法对比,结果显示boosting-monodepth的RMSE值降低了30%,精确度指标δ<1.25时,模型深度信息预测精确度提高了18%,此后以得到的深度图为基础重建管道病害三维点云,并在CloudCompare软件上三维可视化,最后采用随机采样一致算法测算病害深度并和实测数据对比证明其有效性和准确性。

On Robust Cross-view Consistency in Self-supervised Monocular Depth Estimation

Remarkable progress has been made in self-supervised monocular depth estimation (SS-MDE) by exploring cross-view consistency, e.g., photometric consistency and 3D point cloud consistency. However, they are very vulnerable to illumination variance, occlusions, texture-less regions, as well as moving objects, making them not robust enough to deal with various scenes. To address this challenge, we study two kinds of robust cross-view consistency in this paper. Firstly, the spatial offset field between adjacent frames is obtained by reconstructing the reference frame from its neighbors via deformable alignment, which is used to align the temporal depth features via a depth feature alignment (DFA) loss. Secondly, the 3D point clouds of each reference frame and its nearby frames are calculated and transformed into voxel space, where the point density in each voxel is calculated and aligned via a voxel density alignment (VDA) loss. In this way, we exploit the temporal coherence in both depth feature space and 3D voxel space for SS-MDE, shifting the “point-to-point” alignment paradigm to the “region-to-region” one. Compared with the photometric consistency loss as well as the rigid point cloud alignment loss, the proposed DFA and VDA losses are more robust owing to the strong representation power of deep features as well as the high tolerance of voxel density to the aforementioned challenges. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques. Extensive ablation study and analysis validate the effectiveness of the proposed losses, especially in challenging scenes. The code and models are available at https://github.com/sunnyHelen/RCVC-depth.

Light field depth estimation:A comprehensive survey from principles to future

Light Field(LF)depth estimation is an important research direction in the area of computer vision and computational photography,which aims to infer the depth information of different objects in threedimensional scenes by capturing LF data.Given this new era of significance,this article introduces a survey of the key concepts,methods,novel applications,and future trends in this area.We summarize the LF depth estimation methods,which are usually based on the interaction of radiance from rays in all directions of the LF data,such as epipolar-plane,multi-view geometry,focal stack,and deep learning.We analyze the many challenges facing each of these approaches,including complex algorithms,large amounts of computation,and speed requirements.In addition,this survey summarizes most of the currently available methods,conducts some comparative experiments,discusses the results,and investigates the novel directions in LF depth estimation.

An Estimation Method of Stress in Soft Rock Based on In-situ Measured Stress in Hard Rock

The law of variation of deep rock stress in gravitational and tectonic stress fields is analyzed based on the Hoek-Brown strength criterion. In the gravitational stress field,the rocks in the shallow area are in an elastic state and the deep,relatively soft rock may be in a plastic state. However,in the tectonic stress field,the relatively soft rock in the shallow area is in a plastic state and the deep rock in an elastic state. A method is proposed to estimate stress values in coal and soft rock based on in-situ measurements of hard rock. Our estimation method relates to the type of stress field and stress state. The equations of rock stress in various stress states are presented for the elastic,plastic and critical states. The critical state is a special stress state,which indicates the conversion of the elastic to the plastic state in the gravitational stress field and the conversion of the plastic to the elastic state in the tectonic stress field. Two cases stud-ies show that the estimation method is feasible.

Performance Analysis of Disparity for Stereoscopic Image Pairs

Disparity is the geometrical difference between images of a stereoscopic pair. In this paper we give a comprehensive analysis of the statistical characteristics of disparity. Based on experiments, we discuss the relations between disparity, depth and object relation between block size and disparity estimation, and the influence of error criteria on disparity estimation.