基于草图交互的视频摘要方法及认知分析

Video sketch summarization, interaction and cognition analysis

视频是一类重要的视觉媒体,也是人们进行信息交流的重要载体.面向视频数据的高效内容表达,以及自然便捷的用户浏览和搜索等交互操作,本文提出了一种面向视频内容的草图摘要及交互方法.首先提出了面向视频语义的草图化表征方式,利用草图抽象性和概括性等特点,提出语义草图概念,支持对视频内容的语义草图注释,同时提出了草图摘要的优化布局算法.在此基础上,提出了基于草图摘要的草图交互技术,以及支持交互式视频浏览的自然手势操作,并进一步从认知心理学的角度分析了用于视频摘要的草图表征,以及草图交互中各认知单元的作用及相互关系.最后用户评估的实验结果表明,本文所提出的草图摘要以及草图交互方式提高了获取视频主要内容方面的用户效率,减轻了用户的认知负荷.

Video, as one typical digital media, is important for message communication. For efficient video content visualization and natural interaction such as video browsing and searching, we propose a sketch-based video summarization with fluent sketch interaction in this paper. Firstly, we present the sketch representation for video semantics, which takes the advantages of abstractness and generality of sketches. The concept of semantic sketch is proposed, which supports annotating video contents with sketches. Furthermore, an optimized layout algorithm for sketch summarization is presented. Secondly, we present the interaction techniques for sketch summarization and natural sketch gesture operations. From the viewpoint of cognitive psychology, we analyze the sketch representation, as well as the effects and relations of cognitive units in sketch interaction. Finally, user studies show that the proposed sketch summarization and sketch interaction improve user efficiency in terms of acquiring the main video content and reduce users’ cognitive load.