多模态体育视频语义分析

Multi-modal Analysis of Sports Video for Semantics

以足球运动为例提出了一种体育视频语义结构,并提出相应的语义分析框架。视频被分解为纯视频流和音频流两种模态,每种模态均可依次提取和综合出低层内容和中层内容。视频流可根据低层(物理)内容分割为物理镜头,然后根据特定的中间层内容可以确定为语法镜头。音频也可以在物理特征的基础上形成有意义的中间层内容,如解说员兴奋时的声音。最后,根据视频流和音频流的中间层内容,按照足球比赛转播的规律,分析出比赛中的精彩事件,并选取相关的镜头作为反映此事件的序列组合。

A semantic structure of sports video, exampled with soccer, and corresponding framework for semantics analysis are proposed. Video is parsed into pure video stream and audio stream. Video is Segmented into shots according to low/physical features, and then into syntactic shots with the help of specific middle level contents. Audio can be extracted meaningful middle contents, e.g. excited speech of commenter. According to rules of soccer broadcasting, semantics of highlights can be analyzed based on syntactic contents from video and audio streams.