基于RGB-D数据的自适应局部时空特征提取方法

Adaptive Local Spatiotemporal Feature Extraction Based on RGB-D Data

对于一次学习手势识别,噪声和全局经验运动约束严重影响时空特征的精确与充分提取,为此提出了一种融合颜色和深度(RGB-D)信息的自适应局部时空特征提取方法.首先建立连续两灰度帧和两深度帧的金字塔以及相应的光流金字塔作为尺度空间.然后根据灰度和深度光流的水平与垂直方差自适应提取运动感兴趣区域(motion regions of interest,MRo Is).接着仅在MRo Is内检测角点作为兴趣点,当兴趣点的灰度和深度光流同时满足局部运动约束时即为关键点,局部运动约束是在每个MRo I内自适应确定的.最后在改进的梯度运动空间计算SIFT-like描述子.Chalearn数据库上的实验结果表明:提出方法得到了较高的识别准确率,其识别性能优于现已发表的方法.

Noise and global empirical motion constraints seriously affect extracting accurate and sufficient spatiotemporal features for one-shot learning gesture recognition. To tackle the problem,an adaptive local spatiotemporal feature extraction approach with both color and depth （RGB- D ） information fused was proposed. Firstly,pyramids and optical flow pyramids of successive two gray frames and two depth frames were built as scale space. Then,motion regions of interest （ MRoIs） were adaptively extracted according to horizontal and vertical variances of the gray and depth optical flow. Subsequently,corners were just detected as interest points in the MRoIs. These interest points were selected as keypoints only if their optical flow meet adaptive local gray and depth motion constraints. The local motion constraints were adaptively determined in each MRoI. Finally, SIFT-like descriptors were calculated in improved gradient and motion spaces. Experimental results of ChaLearn dataset demonstrate that the proposed approach has higher recognition accuracy that is comparable to the published approaches.