融合Transformer和语义图卷积的三维人体姿态估计方法

为了进一步提升从单目二维人体姿态预测三维人体姿态的方法性能,提出一种融合Transformer和语义图卷积的三维人体姿态估计模型,模型由4个部分组成,Transformer编码网络、语义图卷积编码网络、姿态坐标预测模块和姿态坐标错误回归模块。首先,Transformer编码网络对关节特征进行全局特征编码,以增强人体姿态的全局关联性。其次,语义图卷积编码网络专注于局部关节特征提取,以加强局部关节特征之间的关联性。接下来,姿态坐标预测模块和姿态坐标错误回归模块将关节全局和局部编码特征融合,以增强对三维姿态的准确建模能力。通过在Human3.6M数据集上进行实验表明,方法在估计性能方面取得了较好的改进,以真实的二维人体姿态作为输入,在MPJPE和PA-MPJPE值分别为32.7和25.9 mm,与实验对照方法相比,性能分别提升了3.82%和1.14%。

基于语义图卷积神经网络的三维人体姿态估计

三维人体姿态估计是计算机视觉领域的重要分支,由于深度模糊和自遮挡等问题,现有的三维人体姿态估计方法的准确率较低。因此,在适用于处理人体骨架数据的语义图卷积的基础上,提出了改进的语义图卷积算法,为了提高网络的信息处理能力,提出了一种使用改进的语义图卷积层将特征映射到高维度潜空间并融合不同特征通道信息的网络结构,并在网络末尾添加姿势细化层提高网络的鲁棒性。通过在公开的三维姿态估计数据集Human3.6M上进行实验,并与目前流行的三维姿态估计方法进行对比,验证上述算法有着较高的准确率。

Semantic image segmentation with fused CNN features

Semantic image segmentation is a task to predict a category label for every image pixel. The key challenge of it is to design a strong feature representation. In this paper, we fuse the hierarchical convolutional neural network(CNN) features and the region-based features as the feature representation. The hierarchical features contain more global information, while the region-based features contain more local information. The combination of these two kinds of features significantly enhances the feature representation. Then the fused features are used to train a softmax classifier to produce per-pixel label assignment probability. And a fully connected conditional random field(CRF) is used as a post-processing method to improve the labeling consistency. We conduct experiments on SIFT flow dataset. The pixel accuracy and class accuracy are 84.4% and 34.86%, respectively.