镜头内容分析及其在视频检索中的应用

Shot Content Analysis for Video Retrieval Applications

提出了一种镜头内容分析方法及其在视频检索中的两个应用:镜头检索与场景结构提取.为了刻画一个镜头的内容变化,首先引入两个新的内容描述子:主色直方图和空间结构直方图.主色直方图能够捕捉那些持续时间最长的颜色,而这些颜色是这段视频所关注的对象或背景的主要颜色.从颜色块图提取的空间结构直方图是描述图像空间信息的一组特征.一个变化较大的镜头可以划分为几个内容一致的子镜头,两个镜头的相似性可以从对应子镜头的相似性计算得到.镜头相似性度量可以直接用于镜头检索,还可用于场景结构提取.另外,还提出分裂与合并力量竞争的场景结构提取方法.在大容量视频数据库上进行实验所得结果证实了该方法在镜头检索和场景提取的优异表现.

A scheme on shot content analysis for two video retrieval applications, shot retrieval and scene structure extraction, is presented. To characterize the temporal content variations in one shot, two descriptors: Dominant Cola Histograms and Spatial Structure Histograms, are developed. By fusing temporal information into color content, Dominant Color Histograms for a group of frames are trying to capture the dominant colors with longer duration, which would be the colors of the focused objects or background. Spatial Structure Histograms is a set of features extracted from color-blob maps to describe spatial information for an individual frame. A shot with significant content changes can be segmented into several subshots that are of coherent content, and shot similarity measure can be computed from the similarity between corresponding sub-shots. Scene structure is extracted by analyzing the competition of splitting and merging forces. Experimental results on real-world sports video show that the proposed approaches can achieve the best performance on shot retrievals and have promising results on scene structure extraction.