Depth Map Prediction of Occluded Objects Using Structure Tensor with Gain Regularization

The creation of the 3D rendering model involves the prediction of an accurate depth map for the input images.A proposed approach of a modified semi-global block matching algorithm with variable window size and the gradient assessment of objects predicts the depth map.3D modeling and view synthesis algorithms could effectively handle the obtained disparity maps.This work uses the consistency check method to find an accurate depth map for identifying occluded pixels.The prediction of the disparity map by semi-global block matching has used the benchmark dataset of Middlebury stereo for evaluation.The improved depth map quality within a reasonable process-ing time outperforms the other existing depth map prediction algorithms.The experimental results have shown that the proposed depth map predictioncould identify the inter-object boundaryeven with the presence ofocclusion with less detection error and runtime.We observed that the Middlebury stereo dataset has very few images with occluded objects,which made the attainment of gain cumbersome.Considering this gain,we have created our dataset with occlu-sion using the structured lighting technique.The proposed regularization term as an optimization process in the graph cut algorithm handles occlusion for different smoothing coefficients.The experimented results demonstrated that our dataset had outperformed the Tsukuba dataset regarding the percentage of occluded pixels.

DGConv: A Novel Convolutional Neural Network Approach for Weld Seam Depth Image Detection

We propose a novel image segmentation algorithm to tackle the challenge of limited recognition and segmentation performance in identifying welding seam images during robotic intelligent operations.Initially,to enhance the capability of deep neural networks in extracting geometric attributes from depth images,we developed a novel deep geometric convolution operator(DGConv).DGConv is utilized to construct a deep local geometric feature extraction module,facilitating a more comprehensive exploration of the intrinsic geometric information within depth images.Secondly,we integrate the newly proposed deep geometric feature module with the Fully Convolutional Network(FCN8)to establish a high-performance deep neural network algorithm tailored for depth image segmentation.Concurrently,we enhance the FCN8 detection head by separating the segmentation and classification processes.This enhancement significantly boosts the network’s overall detection capability.Thirdly,for a comprehensive assessment of our proposed algorithm and its applicability in real-world industrial settings,we curated a line-scan image dataset featuring weld seams.This dataset,named the Standardized Linear Depth Profile(SLDP)dataset,was collected from actual industrial sites where autonomous robots are in operation.Ultimately,we conducted experiments utilizing the SLDP dataset,achieving an average accuracy of 92.7%.Our proposed approach exhibited a remarkable performance improvement over the prior method on the identical dataset.Moreover,we have successfully deployed the proposed algorithm in genuine industrial environments,fulfilling the prerequisites of unmanned robot operations.

基于图像关键帧的Visual-Depth Map建立方法

对于室内视觉定位系统,需要在离线阶段建立Visual Map数据库用来存储图像信息,在线阶段用户通过与Visual Map数据库进行比对来完成用户位置的估计.离线阶段建立的数据库可以采用逐点采样或视频流采样的方式.但是无论何种方式,考虑到数据库中图像信息的相似性,传统方式建立的数据库中存储图像有较多冗余,导致增加了在线阶段的定位时间开销.因此,本文根据Visual Map中的相邻图像间的相似性,提出了一种基于图像关键帧的Visual-Depth Map建立方法,有效地减少了离线数据库的规模.在离线阶段,本文使用Kinect传感器同时获得图像信息和深度信息;然后,通过基于图像相似度的图像关键帧算法对原始图像序列进行筛选,得到关键帧序列,从而实现Visual-Depth Map的建立.在线阶段,用户可以直接输入查询图像与Visual-Depth Map中的图像序列进行检索匹配,找到相似度较高的匹配图像,再通过EPnP算法进行2D-3D的位姿估计,完成用户位置的计算.实验证明,本文所提方法可以在保证较高定位精度的前提下,有效减少离线数据库规模,降低在线阶段的定位时间开销.

Depth map软件在网师园空间分析中的应用

空间句法是一种通过对包括建筑、聚落、城市以及景观在内的人居空间结构的量化描述来研究空间组织的理论和方法。作为空间句法的专用分析软件,近年来Depth map在空间结构分析领域得到了越来越广泛的应用。以江南园林网师园为例,利用Depth map软件建立模型,对其空间的可视和可行2个层面分别进行连接度、整合度和深度值的空间结构分析,并得出相应结论:Depth map在定量分析园林空间特性,确定园林空间不同层面之间的联系方面起着重要作用。

An integrated method of selecting environmental covariates for predictive soil depth mapping

Environmental covariates are the basis of predictive soil mapping.Their selection determines the performance of soil mapping to a great extent,especially in cases where the number of soil samples is limited but soil spatial heterogeneity is high.In this study,we proposed an integrated method to select environmental covariates for predictive soil depth mapping.First,candidate variables that may influence the development of soil depth were selected based on pedogenetic knowledge.Second,three conventional methods(Pearson correlation analysis(PsCA),generalized additive models(GAMs),and Random Forest(RF))were used to generate optimal combinations of environmental covariates.Finally,three optimal combinations were integrated to produce a final combination based on the importance and occurrence frequency of each environmental covariate.We tested this method for soil depth mapping in the upper reaches of the Heihe River Basin in Northwest China.A total of 129 soil sampling sites were collected using a representative sampling strategy,and RF and support vector machine(SVM)models were used to map soil depth.The results showed that compared to the set of environmental covariates selected by the three conventional selection methods,the set of environmental covariates selected by the proposed method achieved higher mapping accuracy.The combination from the proposed method obtained a root mean square error(RMSE)of 11.88 cm,which was 2.25–7.64 cm lower than the other methods,and an R^2 value of 0.76,which was 0.08–0.26 higher than the other methods.The results suggest that our method can be used as an alternative to the conventional methods for soil depth mapping and may also be effective for mapping other soil properties.

基于depth-map和分布式视频编码的多视点视频传输方法

针对多视点视频传输系统数据量庞大的问题,提出一种基于深度图(depth-map)和分布式视频编码(DVC)的不等错误保护(UEP)传输方法。该方法首先基于多个视点提取深度图;然后,在传输过程中传输一个视点及其深度图;最后,经过网络传输,在解码端由一个视点图及其深度图生成其他视点。由于视点图和深度图在解码端的重要程度不同,对需要传输的视点图和深度图采用不同的分布式视频编码方法,进行不平等的错误保护。仿真实验结果表明,所提传输方法比传统的分布式多视点视频编码传输系统具有更好的抗误码性能,提高了传输可靠性,图像的峰值信噪比(PSNR)约提高1.5 dB。

INPAINTING ALGORITHM FOR KINECT DEPTH MAP BASED ON FOREGROUND SEGMENTATION

The depth information of the scene indicates the distance between the object and the camera,and depth extraction is a key technology in 3D video system.The emergence of Kinect makes the high resolution depth map capturing possible.However,the depth map captured by Kinect can not be directly used due to the existing holes and noises,which needs to be repaired.We propose a texture combined inpainting algorithm in this paper.Firstly,the foreground is segmented combined with the color characteristics of the texture image to repair the foreground of the depth map.Secondly,region growing is used to determine the match region of the hole in the depth map,and to accurately position the match region according to the texture information.Then the match region is weighted to fill the hole.Finally,a Gaussian filter is used to remove the noise in the depth map.Experimental results show that the proposed method can effectively repair the holes existing in the original depth map and get an accurate and smooth depth map,which can be used to render a virtual image with good quality.

A Sensitivity Mapping Technique for Tensile Force and Case Depth Characterization Based on Magnetic Minor Hysteresis Loops

In the nondestructive testing and evaluation area,magnetic major hysteresis loop measurement technology are widely applied for ferromagnetic material evaluation.However the characterization ability of major hysteresis loop measurement technology greatly varies as the evaluated target properties.To solve this limitation,magnetic minor hysteresis loops,which reflect the responses of ferromagnetic material magnetization in a systematic way,is recommend.Inspired by plenty of information carried by the minor loops,the sensitivity mapping technique was developed to achieve the highest sensitivity of minor-loop parameters to the nondestructively evaluated targets.In this study,for the first time,the sensitivity mapping technique is used to measure the tensile force in a steel strand and evaluate the effective case depth in induction-hardened steel rods.The method and procedures for the sensitivity mapping technique are given before experimental detection.The obtained experimental results indicate that the linear correlation between the induced voltage(or the magnetic induction intensity)and the tensile force(or effective case depth)exists at most of the locations in the cluster of minor loops.The obtained sensitivity maps can be used to optimize the applied magnetic field(or excitation current)and the analyzed locations at the minor loops for achieving the highest sensitivity.For the purpose of tensile force measurement,it is suggested that the strand should be firstly magnetized to the near-saturation state and then restored to the remanent state.In this way,the highest sensitivity is obtained as about 15.26 mV/kN.As for the induction-hardened steel rods,the highest sensitivity of magnetic induction intensity to the effective case depth occurs under low magnetic field conditions and the absolute value of the highest sensitivity is about 0.1110 T/mm.This indicates that if the highest sensitivity is required in the case depth evaluation,the induction-hardened steel rods are only required to be weakly magnetized.The proposed sensitivity mapping technique shows the good performance in the high-sensitivity evaluation of tensile force and case depth in ferromagnetic materials and its application scope can be extended to other nondestructive detection fields.

Edge-Preserving Depth Map Super-Resolution with Intensity Guidance

Depth discontinuity edge affects the visual quality of synthesized images in 3D image warping.However,it suffers from accuracy degradation when up-sampled from low-resolution depth maps,especially at large scaling factors.To preserve the accuracy of depth discontinuity,a novel joint bilateral depth super-resolution with intensity guidance method is proposed.Particularly,the fast local intensity classification is exploited to estimate depth coefficients in joint bilateral up-sampling for depth maps,so as to eliminate depth discontinuity edge misalignment.Additionally,the proposed method is accelerated on graphic processing units(GPUs)to meet the requirement of realtime application.Experiments demonstrate that our method can preserve the accuracy of depth discontinuity edges after super resolution,leveraging the visual quality of synthesized image in 3D image warping.

Combining Multi-scale Directed Depth Motion Maps and Log-Gabor Filters for Human Action Recognition

Recognition of the human actions by computer vision has become an active research area in recent years. Due to the speed and the high similarity of the actions, the current algorithms cannot get high recognition rate. A new recognition method of the human action is proposed with the multi-scale directed depth motion maps(MsdDMMs) and Log-Gabor filters. According to the difference between the speed and time order of an action, MsdDMMs is proposed under the energy framework. Meanwhile, Log-Gabor is utilized to describe the texture details of MsdDMMs for the motion characteristics. It can easily satisfy both the texture characterization and the visual features of human eye. Furthermore, the collaborative representation is employed as action recognition by the classification. Experimental results show that the proposed algorithm, which is applied in the MSRAction3 D dataset and MSRGesture3 D dataset, can achieve the accuracy of 95.79% and 96.43% respectively. It also has higher accuracy than the existing algorithms, such as super normal vector(SNV), hierarchical recurrent neural network(Hierarchical RNN).

Technique of Embedding Depth Maps into 2D Images

This paper proposes a new technique that is used to embed depth maps into corresponding 2-dimensional （2D） images. Since a 2D image and its depth map are integrated into one type of image format, they can be treated as if they were one 2D image. Thereby, it can reduce the amount of data in 3D images by half and simplify the processes for sending them through networks because the synchronization between images for the left and right eyes becomes unnecessary. We embed depth maps in the quantized discrete cosine transform （DCT） data of 2D images. The key to this technique is whether the depth maps could be embedded into 2D images without perceivably deteriorating their quality. We try to reduce their deterioration by compressing the depth map data by using the differences from the next pixel to the left. We assume that there is only one non-zero pixel at most on one horizontal line in the DCT block because the depth map values change abruptly. We conduct an experiment to evaluate the quality of the 2D images embedded with depth maps and find that satisfactory quality could be achieved.

A Fast Depth-Map Generation Algorithm based on Motion Search from 2D Video Contents

Generation of a depth-map from 2D video is the kernel of DIBR (Depth Image Based Rendering) in 2D-3D video conversion systems. However it occupies over most of the system resource where the motion search module takes up 90% time-consuming in typical motion estimation-based depth-map generation algorithms. In order to reduce the computational complexity, in this paper a new fast depth-map generation algorithm based on motion search is developed, in which a fast diamond search algorithm is adopted to decide whether a 16x16 or 4x4 block size is used based on Sobel operator in the motion search module to obtain a sub-depth-map. Then the sub-depth-map will be fused with the sub-depth-maps gotten from depth from color component Cr and depth from linear perspective modules to compensate and refine detail of the depth-map, finally obtain a better depth-map. The simulation results demonstrate that the new approach can greatly reduce over 50% computational complexity compared to other existing methods.

Generate Reservoir Depths Mapping by Using Digital Elevation Model: A Case Study of Mosul Dam Lake, Northern Iraq

Not many bathymetric maps are available for many lakes and reservoirs in developing countries. Usually the bathymetric mapping requires investment in expensive equipment and fieldwork, both of which are not accessible in these countries. This work demonstrates the ability to develop bathymetric map of Mosul Lake by using a digital elevation model (DEM). The depths model of the lake was designed through the use of three main stages;a coastline extraction, dataset interpolation and a triangular irregular network model. The normalized difference water index (NDWI) was used for automatic delineation of the lake coastline from satellite images. The ordinary kriging interpolation with a stable model was used to interpolate the water depths dataset. Finally a triangulated irregular network (TIN) model was used to visualize the resulting interpolation model. Calculated values of area and volume of a TIN model during 2011 were compared with values of supposed initial operation of the reservoir. The differences of water volume storage between these stages at 321 m water level was about 0.81 × 109 m3, where the lake lost around 10% of storage value. Also the results of depths lake model show that the change in water storage between March and July 2011 was about 3.08 × 109 m3.

3DV quality model based depth maps for view synthesis in FTV system

Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts in the synthesized views. To solve this problem, a 3D video quality model base depth maps （D-3DV） for virtual view synthesis and depth map coding in the FTV applications is proposed. First, the relationships between distortions in coded depth map and rendered view are derived. Then, a precisely 3DV quality model based depth characteristics is develop for the synthesized virtual views. Finally, based on D-3DV model, a multilateral filtering is applied as a pre-processed filter to reduce rendering artifacts. The experimental results evaluated by objective and subjective methods indicate that the proposed D-3DV model can reduce bit-rate of depth coding and achieve better rendering quality.

基于映射空间编码的高速运动轨道图像去模糊研究

针对轨道缺陷检测系统因镜头抖动或相机快速移动而导致所采集图像较为模糊的问题,提出一种基于最大后验概率估计思想的映射空间编码的高速运动轨道图像去模糊算法。首先,该算法使用深度编解码器和残差网络分别对数据集中清晰图像到模糊图像的映射关系和模糊核进行编码,为了保证编码时频率信息的完整性,算法在传统的残差模块上引入快速傅里叶变换通道构成双通道残差网络,以补偿多次特征提取带来的频率损失;其次,算法采用深度图像先验(Deep Image Prior,DIP)将潜在的清晰图像和模糊核进行参数化,再利用先验得到的模糊核和清晰图像来调用编码空间中的映射关系;最后,通过交替优化潜在的清晰图像和模糊核,从而去逼近一个真实未知的映射,进而实现真实场景下高速运动轨道图像的去模糊。实验结果表明,双通道残差模块提取的特征图频率信息分量强度普遍高于传统的残差模块,相较于使用传统残差模块实现该算法,采用双通道残差模块可使峰值信噪比(Peak Signal-to-Noise Ratio,PSNR)提升0.84 dB,结构相似性(Structural Similarity,SSIM)提高0.025 1。与现有的深度学习去模糊算法相比,提出的去模糊算法对高速轨道检测系统所采集图像的去模糊效果更佳,在性能方面相较于最好的去模糊算法,PSNR提高了1.84 dB,SSIM提升了0.017 3,显著提升了采集图像的质量。研究结果可为下一步识别轨道部件是否存在缺陷提供清晰图像。

基于深度强化学习的移动机器人三维路径规划方法

针对三维未知环境中存在的高复杂度和不确定性的问题,提出一种在有限观测空间优化策略下基于深度强化学习的移动机器人三维路径规划方法。首先,在有限观测空间下采用深度图信息作为智能体的输入,模拟移动受限且未知的复杂三维空间环境;其次,设计了两阶段离散动作空间下的动作选择策略,包括方向动作和位移动作,以减少搜索步数和时间;最后,在近端策略优化(PPO)算法基础上,添加门控循环单元(GRU)结合历史状态信息,以提升未知环境中搜索策略的稳定性,进而提高规划路径准确度和平滑度。实验结果表明,相较于A2C(Advantage ActorCritic),所提方法的平均搜索时间缩短了49.07%,平均规划路径长度缩短了1.04%,同时能够完成线性时序逻辑约束下的多目标路径规划任务。

基于区域地质图与数字高程模型的地层埋深预测方法及应用

对于页岩气、水溶气等非常规天然气资源早期选区评价,往往因为缺乏地震资料导致难以依赖地震构造解释获得目的层的埋深,故借助其他非地震资料有效预测目的层区域性埋深具有重要意义。区域地质图与数字高程模型(DEM)都是广泛覆盖且容易获取的基础性资料。利用区域地质图蕴含的地层产状、地下构造起伏趋势、地层埋深和地表出露地层年代的关系等信息,再叠合DEM地表高程,建立了一种有效预测目的层构造高度和埋深的新方法。利用四川盆地及周缘地区36幅1∶20万区域地质图和美国航空航天局(NASA)数字高程模型资料,预测了该区下志留统龙马溪组底界埋深,据此编制的龙马溪组底界埋深等值线图为该区页岩气保存条件评价提供了重要参数。该方法不仅可为页岩气早期选区评价提供有效支撑,还可应用于深部高压含水层中水溶气的资源评价、深部咸含水层CO_(2)地质封存的构造优选。

基于锚点的快速三维手部关键点检测算法

在人机协作任务中,手部关键点检测为机械臂提供目标点坐标,A2J(Anchor-to-Joint)是具有代表性的一种利用锚点进行关键点检测的方法。A2J以深度图为输入,可实现较好的检测效果,但对全局特征获取能力不足。文中设计了全局-局部特征融合模块(Global-Local Feature Fusion,GLFF)对骨干网络浅层和深层的特征进行融合。为了提升检测速度,文中将A2J的骨干网络替换为ShuffleNetv2并对其进行改造,用5×5深度可分离卷积替换3×3深度可分离卷积,增大感受野,有效提升了骨干网络对全局特征的提取能力。文中在锚点权重估计分支引入高效通道注意力模块(Efficient Channel Attention,ECA),提升了网络对重要锚点的关注度。在主流数据集ICVL和NYU上进行的训练和测试结果表明,相比于A2J,文中所提方法的平均误差分别降低了0.09 mm和0.15 mm。在GTX1080Ti显卡上实现了151 frame·s^(-1)的检测速率,满足人机协作任务对于实时性的要求。

基于激光雷达与相机融合的树干检测方法

针对传统传感器在树干检测中的局限性和单一性,提出一种基于激光雷达与相机融合的树干检测方法.首先,利用深度图对激光雷达点云进行处理,实现地面点云去除以及树干点云聚类,并在聚类中设置横、纵向自适应阈值,去除聚类中墙体、杂草、树叶等多余信息;然后,利用YOLOv3算法对相机图像进行分析,基于树干特征实现目标识别并返回检测框与类别信息;最后,基于交并比方法(IoU)对2种传感器的检测结果进行融合,识别树干并返回其三维信息与位置信息.以无人割草机为载体开展场地测试,实验结果表明:融合算法的树干检测准确率在93.1%左右,树干定位横、纵向平均误差分别为0.075 m和0.078 m,能够满足无人割草机的树干检测要求,为智能农机的环境感知提供了一种新的方法.

融合单目深度和 RTK 定位的电力线弧垂测量方法

现有电力线路弧垂测量方法操作繁琐且智能化程度低,提出一种融合单目深度和RTK定位的电力线弧垂测量方法。首先,无人机拍摄电力线路关键点图像,将其输入构建的单目深度估计模型EleDep-Net生成对应深度图,该模型嵌入带状金字塔模块和边界融合注意力模块,使模型能精准地捕捉导线上下文语义信息;其次,引入深度矫正算法进一步修正深度图中的深度值,根据修正后的深度值得到关键点深度信息;最后,结合无人机RTK定位和关键点深度信息,在参考坐标系中生成关键点空间坐标进而拟合出导线抛物线公式,通过该公式计算出导线弧垂。在配网线路真实环境中进行测试,结果表明本方法在保证测量相对误差小于5%的前提下,作业效率明显提高,有较高的工程应用价值。