融合时空多特征表示的无监督视频分割算法

Unsupervised video segmentation by fusing multiple spatio-temporal feature representations

针对视频分割的难点在于分割目标的无规则运动、快速变换的背景、目标外观的任意变化与形变等,提出了一种基于时空多特征表示的无监督视频分割算法,通过融合像素级、超像素级以及显著性三类特征设计由细粒度到粗粒度的稳健特征表示。首先,采用超像素分割对视频序列进行处理以提高运算效率,并设计图割算法进行快速求解;其次,利用光流法对相邻帧信息进行匹配,并通过K-D树算法实现最近邻搜索以引入各超像素的非局部时空颜色特征,从而增强分割的鲁棒性;然后,对采用超像素计算得到的分割结果,设计混合高斯模型进行完善;最后,引入图像的显著性特征,协同超像素分割与混合高斯模型的分割结果,设计投票获得更加准确的视频分割结果。实验结果表明,所提算法是一种稳健且有效的分割算法,其结果优于当前大部分无监督视频分割算法及部分半监督视频分割算法。

Due to random movement of the segmented target, rapid change of background, arbitrary variation and shape deformation of object appearance, in this paper, a new unsupervised video segmentation algorithm based on multiple spatialtemporal feature representations was presented. By combination of salient features and other features obtained from pixels and superpixels, a coarse-to-fine-grained robust feature representation was designed to represent each frame in a video sequence.Firstly, a set of superpixels was generated to represent foreground and background in order to improve computational efficiency and get segmentation results by graph-cut algorithm. Then, the optical flow method was used to propagate information between adjacent frames, and the appearance of each superpixel was updated by its non-local sptatial-temporal features generated by nearest neighbor searching method with efficient K-Dimensional tree( K-D tree) algorithm, so as to improve robustness of segmentation. After that, for segmentation results generated in superpixel-level, a new Gaussian mixture model based on pixels was constructed to achieve pixel-level refinement. Finally, the significant feature of image was introduced, as well as segmentation results generated by graph-cut and Gaussian mixture model, to obtain more accurate segmentation results by voting scheme. The experimental results show that the proposed algorithm is a robust and effective segmentation algorithm,which is superior to most unsupervised video segmentation algorithms and some semi-supervised video segmentation algorithms.