作为人工智能计算机视觉领域一项重要的任务,3D人体姿态估计受到了广泛的关注,并成功地应用在人机交互、电影游戏制作等领域。然而,3D人体姿态估计仍然面临着很大的挑战,主要是人体遮挡问题和数据集视角冗余问题,这些问题严重影响了3D...作为人工智能计算机视觉领域一项重要的任务,3D人体姿态估计受到了广泛的关注,并成功地应用在人机交互、电影游戏制作等领域。然而,3D人体姿态估计仍然面临着很大的挑战,主要是人体遮挡问题和数据集视角冗余问题,这些问题严重影响了3D人体姿态估计结果精度与速度的提升。本文提出了一种基于多特征提取的3D人体姿态估计方法。首先通过采集多个相机视角下的图片数据,将所采图片数据放入2D人体关节点检测网络模型中,得到人体2D关节点。接着将采集到的人体数据输入到关节点置信度计算网络模型,得到视角图片中各个关节点的权重值。随后将2D人体关节点热图通过一个热图权重计算网络计算出热图权重,将各个视角下的权重特征计算融合得到加权后的2D人体关节点热图。最后将所得加权后的2D人体关节点热图和视角图片中各个关节点的权重值输入到三角化算法中,映射得到空间中的3D人体关节点。本文的关键思想是设计一个关节点置信度计算网络从输入图像中学习每个关节的置信度权重,同时提取了反映热图特征质量的权重矩阵,以提高遮挡视图中热图的特征质量。此外,使用感知哈希算法对Occlusion-Person数据集进行去视角实验,在保证结果准确性的同时提高了模型推理速度。本文方法是端到端可微的,可以显著地提高算法效率和鲁棒性。本文在Human3.6M和Occlusion-Person两个公共数据集上使用平均关节位置误差(Mean Per Joint Position Error,MPJPE)指标对该方法进行评估,分别取得27.3 mm和9.7 mm的结果。实验结果表明,该算法与最先进的方法相比,性能有了显著提升。展开更多
A 3D-view is helpful to instantly grasp what is presented in a drawing. There exist a variety of ways to present the same part with 3D-views. To facilitate the choice of an optimum one among them, the work divides com...A 3D-view is helpful to instantly grasp what is presented in a drawing. There exist a variety of ways to present the same part with 3D-views. To facilitate the choice of an optimum one among them, the work divides composite solid models into three categories, so as to convey the originality of design concisely and accurately by using the least ＂ engineering language＂.展开更多
针对多视图立体网络在弱纹理或非朗伯曲面等挑战性区域重建效果差的问题,首先提出一个基于3个并行扩展卷积和注意力机制的多尺度特征提取模块,在增加感受野的同时捕获特征之间的依赖关系以获取全局上下文信息,从而提升多视图立体网络在...针对多视图立体网络在弱纹理或非朗伯曲面等挑战性区域重建效果差的问题,首先提出一个基于3个并行扩展卷积和注意力机制的多尺度特征提取模块,在增加感受野的同时捕获特征之间的依赖关系以获取全局上下文信息,从而提升多视图立体网络在挑战性区域特征的表征能力以进行鲁棒的特征匹配。其次在代价体正则化3D CNN部分引入注意力机制,使网络注意于代价体中的重要区域以进行平滑处理。另外建立一个神经渲染网络,该网络利用渲染参考损失精确地解析辐射场景表达的几何外观信息,并引入深度一致性损失保持多视图立体网络与神经渲染网络之间的几何一致性,有效地缓解有噪声代价体对多视图立体网络的不利影响。该算法在室内DTU数据集中测试,点云重建的完整性和整体性指标分别为0.289和0.326,与基准方法CasMVSNet相比,分别提升24.9%和8.2%,即使在挑战性区域也得到高质量的重建效果;在室外Tanks and Temples中级数据集中,点云重建的平均F-score为60.31,与方法UCS-Net相比提升9.9%,体现出较强的泛化能力。展开更多
Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts ...Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts in the synthesized views. To solve this problem, a 3D video quality model base depth maps （D-3DV） for virtual view synthesis and depth map coding in the FTV applications is proposed. First, the relationships between distortions in coded depth map and rendered view are derived. Then, a precisely 3DV quality model based depth characteristics is develop for the synthesized virtual views. Finally, based on D-3DV model, a multilateral filtering is applied as a pre-processed filter to reduce rendering artifacts. The experimental results evaluated by objective and subjective methods indicate that the proposed D-3DV model can reduce bit-rate of depth coding and achieve better rendering quality.展开更多
针对在基于深度学习技术的特征提取网络中,深层次的卷积神经网络提取的特征缺乏低级语义信息的问题,该文提出了语义增强的多视立体视觉方法。首先,提出了一种ConvLSTM(Convolutional Long Short-Term Memory)语义聚合网络,通过使用ConvL...针对在基于深度学习技术的特征提取网络中,深层次的卷积神经网络提取的特征缺乏低级语义信息的问题,该文提出了语义增强的多视立体视觉方法。首先,提出了一种ConvLSTM(Convolutional Long Short-Term Memory)语义聚合网络,通过使用ConvLSTM网络结构,对多个卷积层提取的特征图进行预测,得到融合每层语义信息的特征图,有助于在空间上层层抽取图像的高级特征时,利用长短期记忆神经网络结构的记忆功能来增强高层特征图中的低级语义信息,提高了弱纹理区域的重建效果,提高了3D重建的鲁棒性和完整性;其次,提出了一种可见性网络,在灰度图的基础上,通过突出特征图上可见区域的特征,加深了可见区域在特征图中的影响,有助于提高三维重建效果;最后,提取图像的纹理信息,并进入ConvLSTM语义聚合网络提取深层次特征,提高了弱纹理区域的重建效果。与主流的多视立体视觉重建方法相比,重建效果较好。展开更多
The growing demand for current and precise geographic information that pertains to urban areas has given rise to a significant interest in digital surface models that exhibit a high level of detail. Traditional method...The growing demand for current and precise geographic information that pertains to urban areas has given rise to a significant interest in digital surface models that exhibit a high level of detail. Traditional methods for creating digital surface models are insufficient to reflect the details of earth’s features. These models only represent three-dimensional objects in a single texture and fail to offer a realistic depiction of the real world. Furthermore, the need for current and precise geographic information regarding urban areas has been increasing significantly. This study proposes a new technique to address this problem, which involves integrating remote sensing, Geographic Information Systems (GIS), and Architecture Environment software environments to generate a detailed three-dimensional model. The processing of this study starts with: 1) Downloading high-resolution satellite imagery; 2) Collecting ground truth datasets from fieldwork; 3) Imaging nose removing; 4) Generating a Two-dimensional Model to create a digital surface model in GIS using the extracted building outlines; 5) Converting the model into multi-patch layers to construct a 3D model for each object separately. The results show that the 3D model obtained through this method is highly detailed and effective for various applications, including environmental studies, urban development, expansion planning, and shape understanding tasks.展开更多
文摘作为人工智能计算机视觉领域一项重要的任务,3D人体姿态估计受到了广泛的关注,并成功地应用在人机交互、电影游戏制作等领域。然而,3D人体姿态估计仍然面临着很大的挑战,主要是人体遮挡问题和数据集视角冗余问题,这些问题严重影响了3D人体姿态估计结果精度与速度的提升。本文提出了一种基于多特征提取的3D人体姿态估计方法。首先通过采集多个相机视角下的图片数据,将所采图片数据放入2D人体关节点检测网络模型中,得到人体2D关节点。接着将采集到的人体数据输入到关节点置信度计算网络模型,得到视角图片中各个关节点的权重值。随后将2D人体关节点热图通过一个热图权重计算网络计算出热图权重,将各个视角下的权重特征计算融合得到加权后的2D人体关节点热图。最后将所得加权后的2D人体关节点热图和视角图片中各个关节点的权重值输入到三角化算法中,映射得到空间中的3D人体关节点。本文的关键思想是设计一个关节点置信度计算网络从输入图像中学习每个关节的置信度权重,同时提取了反映热图特征质量的权重矩阵,以提高遮挡视图中热图的特征质量。此外,使用感知哈希算法对Occlusion-Person数据集进行去视角实验,在保证结果准确性的同时提高了模型推理速度。本文方法是端到端可微的,可以显著地提高算法效率和鲁棒性。本文在Human3.6M和Occlusion-Person两个公共数据集上使用平均关节位置误差(Mean Per Joint Position Error,MPJPE)指标对该方法进行评估,分别取得27.3 mm和9.7 mm的结果。实验结果表明,该算法与最先进的方法相比,性能有了显著提升。
文摘A 3D-view is helpful to instantly grasp what is presented in a drawing. There exist a variety of ways to present the same part with 3D-views. To facilitate the choice of an optimum one among them, the work divides composite solid models into three categories, so as to convey the originality of design concisely and accurately by using the least ＂ engineering language＂.
文摘针对多视图立体网络在弱纹理或非朗伯曲面等挑战性区域重建效果差的问题,首先提出一个基于3个并行扩展卷积和注意力机制的多尺度特征提取模块,在增加感受野的同时捕获特征之间的依赖关系以获取全局上下文信息,从而提升多视图立体网络在挑战性区域特征的表征能力以进行鲁棒的特征匹配。其次在代价体正则化3D CNN部分引入注意力机制,使网络注意于代价体中的重要区域以进行平滑处理。另外建立一个神经渲染网络,该网络利用渲染参考损失精确地解析辐射场景表达的几何外观信息,并引入深度一致性损失保持多视图立体网络与神经渲染网络之间的几何一致性,有效地缓解有噪声代价体对多视图立体网络的不利影响。该算法在室内DTU数据集中测试,点云重建的完整性和整体性指标分别为0.289和0.326,与基准方法CasMVSNet相比,分别提升24.9%和8.2%,即使在挑战性区域也得到高质量的重建效果;在室外Tanks and Temples中级数据集中,点云重建的平均F-score为60.31,与方法UCS-Net相比提升9.9%,体现出较强的泛化能力。
基金supported by the National Natural Science Foundation of China(Grant No.60832003)Key Laboratory of Advanced Display and System Application(Shanghai University),Ministry of Education,China(Grant No.P200902)the Key Project of Science and Technology Commission of Shanghai Municipality(Grant No.10510500500)
文摘Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts in the synthesized views. To solve this problem, a 3D video quality model base depth maps （D-3DV） for virtual view synthesis and depth map coding in the FTV applications is proposed. First, the relationships between distortions in coded depth map and rendered view are derived. Then, a precisely 3DV quality model based depth characteristics is develop for the synthesized virtual views. Finally, based on D-3DV model, a multilateral filtering is applied as a pre-processed filter to reduce rendering artifacts. The experimental results evaluated by objective and subjective methods indicate that the proposed D-3DV model can reduce bit-rate of depth coding and achieve better rendering quality.
文摘针对在基于深度学习技术的特征提取网络中,深层次的卷积神经网络提取的特征缺乏低级语义信息的问题,该文提出了语义增强的多视立体视觉方法。首先,提出了一种ConvLSTM(Convolutional Long Short-Term Memory)语义聚合网络,通过使用ConvLSTM网络结构,对多个卷积层提取的特征图进行预测,得到融合每层语义信息的特征图,有助于在空间上层层抽取图像的高级特征时,利用长短期记忆神经网络结构的记忆功能来增强高层特征图中的低级语义信息,提高了弱纹理区域的重建效果,提高了3D重建的鲁棒性和完整性;其次,提出了一种可见性网络,在灰度图的基础上,通过突出特征图上可见区域的特征,加深了可见区域在特征图中的影响,有助于提高三维重建效果;最后,提取图像的纹理信息,并进入ConvLSTM语义聚合网络提取深层次特征,提高了弱纹理区域的重建效果。与主流的多视立体视觉重建方法相比,重建效果较好。
文摘The growing demand for current and precise geographic information that pertains to urban areas has given rise to a significant interest in digital surface models that exhibit a high level of detail. Traditional methods for creating digital surface models are insufficient to reflect the details of earth’s features. These models only represent three-dimensional objects in a single texture and fail to offer a realistic depiction of the real world. Furthermore, the need for current and precise geographic information regarding urban areas has been increasing significantly. This study proposes a new technique to address this problem, which involves integrating remote sensing, Geographic Information Systems (GIS), and Architecture Environment software environments to generate a detailed three-dimensional model. The processing of this study starts with: 1) Downloading high-resolution satellite imagery; 2) Collecting ground truth datasets from fieldwork; 3) Imaging nose removing; 4) Generating a Two-dimensional Model to create a digital surface model in GIS using the extracted building outlines; 5) Converting the model into multi-patch layers to construct a 3D model for each object separately. The results show that the 3D model obtained through this method is highly detailed and effective for various applications, including environmental studies, urban development, expansion planning, and shape understanding tasks.
