Real-time flow-based video abstraction using OpenCL

A non-photorealistic rendering technique is a method to show various effects different from those of realistic image generation.Of the various techniques,flow-based image abstraction displays the shape and color features well and performs a stylistic visual abstraction.But real-time rendering is impossible when CPU is used because it applies various filtering and iteration methods.In this paper,we present real-time processing methods of video abstraction using open open computing language(OpenCL),technique of general-purpose computing on graphics processing units(GPGPU).Through the acceleration of general-purpose computing(GPU),16 frame-per-second(FPS)or greater is shown to process video abstraction.

Interactive transport of multi-view videos for 3DTV applications

The authors propose a novel method for transporting multi-view videos that aims to keep the bandwidth requirements on both end-users and servers as low as possible. The method is based on application layer multicast, where each end point re- ceives only a selected number of views required for rendering video from its current viewpoint at any given time. The set of selected videos changes in real time as the user’s viewpoint changes because of head or eye movements. Techniques for reducing the black-outs during fast viewpoint changes were investigated. The performance of the approach was studied through network experiments.

Multi-view video color correction using dynamic programming

Color inconsistency between views is an important problem to be solved in multi-view video systems. A multi-view video color correction method using dynamic programming is proposed. Three-dimensional histograms are constructed with sequential conditional probability in HSI color space. Then, dynamic programming is used to seek the best color mapping relation with the minimum cost path between target image histogram and source image histogram. Finally, video tracking technique is performed to correct multi-view video. Experimental results show that the proposed method can obtain better subjective and objective performance in color correction.

Color compensation for multi-view video coding based on diversity of cameras

A novel color compensation method for multi-view video coding (MVC) is proposed, which efficiently exploits the inter-view dependencies between views with the existence of color mismatch caused by the diversity of cameras. A color compensation model is developed in RGB channels and then extended to YCbCr channels for practical use. A modified inter-view reference picture is constructed based on the color compensation model, which is more similar to the coding picture than the original inter-view reference picture. Moreover, the color compensation factors can be derived in both encoder and decoder, therefore no additional data need to be transmitted to the decoder. The experimental results show that the proposed method improves the coding efficiency of MVC and maintains good subjective quality.

Frame-layer bit allocation for multi-view video coding based on frame complexity estimation

Current multi-view video coding （MVC） reference model in joint video team （JVT） does not provide efficient rate control schemes. This paper presents a rate control algorithm for MVC by improving the quadratic rate-distortion （R-D） model. We reasonably allocate bit-rate among views based on the correlation analysisl The proposed algorithm consists of three levels to control the rate bits more accurately, of which the frame layer allocates bits according to the frame complexity and the temporal activity. Extensive experiments show that the proposed algorithm can control the bit rate efficiently.

Efficient fast mode decision using mode complexity for multi-view video coding

The variable block-size motion estimation(ME) and disparity estimation(DE) are adopted in multi-view video coding(MVC) to achieve high coding efficiency. However, much higher computational complexity is also introduced in coding system, which hinders practical application of MVC. An efficient fast mode decision method using mode complexity is proposed to reduce the computational complexity. In the proposed method, mode complexity is firstly computed by using the spatial, temporal and inter-view correlation between the current macroblock(MB) and its neighboring MBs. Based on the observation that direct mode is highly possible to be the optimal mode, mode complexity is always checked in advance whether it is below a predefined threshold for providing an efficient early termination opportunity. If this early termination condition is not met, three mode types for the MBs are classified according to the value of mode complexity, i.e., simple mode, medium mode and complex mode, to speed up the encoding process by reducing the number of the variable block modes required to be checked. Furthermore, for simple and medium mode region, the rate distortion(RD) cost of mode 16×16 in the temporal prediction direction is compared with that of the disparity prediction direction, to determine in advance whether the optimal prediction direction is in the temporal prediction direction or not, for skipping unnecessary disparity estimation. Experimental results show that the proposed method is able to significantly reduce the computational load by 78.79% and the total bit rate by 0.07% on average, while only incurring a negligible loss of PSNR(about 0.04 d B on average), compared with the full mode decision(FMD) in the reference software of MVC.

CODING-ORIENTED MULTI-VIEW VIDEO COLOR CORRECTION

Color inconsistency between views is an important problem to be solved in multi-view video applications, such as free viewpoint television and other three-dimensional video systems. In this paper, by combining with multi-view video coding, a coding-oriented multi-view video color correction method is proposed. We first separate foreground and background in first Group Of Pictures （GOP） by using SKIP coding mode. Then by transferring means and standard deviations in backgrounds, color correction is performed for each frame in GOP, and multi-view video coding is performed and used to renew the backgrounds. Experimental results ances in color correction and multi-view video show the proposed method can obtain better performcoding.

Trifocal tensor based side information generation for multi-view distributed video code

Distributed video coding （DVC） is a new video coding approach based on Wyner-Ziv theorem. The novel uplink-friendly DVC, which offers low-complexity, low-power consuming, and low-cost video encoding, has aroused more and more research interests. In this paper a new method based on multiple view geometry is presented for spatial side information generation of uncalibrated video sensor network. Trifocal tensor encapsulates all the geometric relations among three views that are independent of scene structure; it can be computed from image correspondences alone without requiring knowledge of the motion or calibration. Simulation results show that trifocal tensor-based spatial side information improves the rate-distortion performance over motion compensation based interpolation side information by a maximum gap of around 2dB. Then fusion merges the different side information （temporal and spatial） in order to improve the quality of the final one. Simulation results show that the rate-distortion gains about 0.4 dB.

Bandwidth-Efficient Transmission Method for User View-Oriented Video Services

The trend in video viewing has been evolving beyond simply providing a multi-view option.Recently,a function that allows selection and viewing of a clip from a multi-view service that captures a specific range or object has been added.In particular,the free-view service is an extended concept of multi-view and provides a freer viewpoint.However,since numerous videos and additional data are required for its construction,all of the clips constituting the content cannot be simultaneously provided.Only certain clips are selected and provided to the user.If the video is not the preferred video,change request is made,and a delay occurs during retransmission from the server.Delays due to frequent re-requests degrade the overall quality of service.For free-view services,selectively transmitting the video according to the user’s desired viewpoint and region of interest within the limited network of available videos is important.In this study,we propose a method of screening and providing the correct video based on objects in the contents.Based on the method of recognizing the object in each clip,we designed a method of setting its priority based on information about the object’s location for each viewpoint.During the transmission and receiving process using this information,the selected video can be rapidly recognized and changed.Herein,we present a service system configuration method and propose video selection examples for free-view services.

利用情感激励提取足球视频精彩镜头

足球视频精彩镜头的提取是足球视频检索和摘要生成的关键技术,有着很高的学术研究价值和广泛的应用前景.在系统分析现有足球视频精彩镜头提取方法的基础上,从观众情绪波动的视角出发,以Hanjalic情感曲线的理论思想为基础,设计了一个基于情感激励模型提取足球视频精彩镜头的系统.针对足球视频构建了一种新的特征——镜头激烈度,用此特征取代运动强度特征建立情感激励模型,从而改进了Hanjalic情绪激励曲线的生成方法,提高了系统的查全率、准确率和计算性能.结合足球领域知识,实现了更准确的精彩镜头定位方法,可以根据用户的实际观看时间自动筛选精彩镜头.实验结果验证了算法具有很好的检测效果.

视频摘要技术综述

基于内容的视频分析、检索与索引技术是当前多媒体领域研究的热点之一。视频摘要主要分为两类:静态视频摘要模式(视频略览),包括基于MPEG-1/2的关键帧表示和MPEG-4的关键对象表示;动态视频摘要模式(视频梗概),包括视频概述和精彩片段。该文介绍了视频摘要的新方向——基于情感、心理学等视频摘要模型。

层次化新闻视频处理框架的设计与实现

提出了一个通用的层次化新闻视频处理框架,将新闻视频处理分为句法分段、语义标注以及视频摘要三个层次,并给出了三个层次中涉及的故事单元探测、字幕探测、视频摘要等关键技术的解决方案。框架突破了传统的新闻视频处理框架仅局限于句法分段以及单媒体特征进行处理的缺陷,通过对视音频特征进行多模态的综合分析来获取新闻视频高层的语义内容。实验通过一个新闻视频处理原型系统NVPS验证了框架的可行性,重点对故事单元探测、标题探测以及口播帧探测三个算法进行了实验,实验结果分别达到88%,86%和86%的探测准确率,从而进一步证实了层次框架在新闻视频处理方面的有效性。

基于目标检测及跟踪的视频摘要技术研究

为了能够在不丢失有用信息的前提下对海量监控视频进行摘要,提出一种基于目标检测及跟踪的视频摘要技术。首先根据混合高斯模型对视频进行背景建模,获得监控视频的背景;然后通过背景减法对运动目标进行检测,得到前景运动目标;其次对检测到的运动目标采用分层关联的思想进行目标跟踪,得到运动目标的完整信息;最后将这些运动目标及视频背景重新组装成摘要视频。实验结果表明,该方法能够有效地对监控视频进行浓缩,形成的摘要视频能够完整地保存原有视频信息,减少了存储空间,降低了成本,方便了相关人员及时获取有用信息,提高了工作效率。

视频摘要技术研究

对视频摘要技术进行了研究,将视频摘要按其表现形式分为以标题、关键帧和故事板为代表的静态视频摘要以及以缩略视频为代表的动态视频摘要,并对其中涉及的关键技术进行了探讨,最后对视频摘要技术未来的发展方向进行了总结与展望。

一种新的动态视频摘要生成方法

本文提出了一种概要序列形式的视频摘要生成方法.在视频结构化的基础上,从减少信息冗余和保持本质内容两个角度出发,分别定义了镜头(场景)的冗余度和语义重要度,据此推导出视频内容缩减和保持的原则;之后,综合可理解性的要求,给出了一种自上而下的视频摘要生成方法.以电影动作片为例的实验结果证实了方法的有效性.

面向事件的影片摘要生成方法

对视频摘要的研究已成为视频应用领域十分活跃的课题。为了获得更加实用的视频摘要,在介绍视频摘要概念和用途的基础上,依据目前的研究状况和影片类视频的特点,提出了一种适合于故事类影片的面向事件的影片摘要生成方法,并首先对该方法的细节进行了描述。该方法还使用了一种场景重要程度的评价算法,此算法的特点在于综合利用了影片中的多种媒体特征,其不仅考虑了运动特性,还考虑了感人内容对场景重要程度的影响。通过对摘要进行可用度测试和分析的结果,表明,此种方法生成的影片摘要具有较好的可用度。

基于关键帧的视频摘要研究

提出基于DirectShow组件结构实时提取各类音视频特征的方法,并综合利用这些特征,通过对常用的视频关键帧提取算法进行分析比较,提出了基于FCNN模型的自适应关键帧提取方法,该方法可以适应于各种视频类型。以关键帧为基础,提出利用插值算法建立视频摘要的方法。最后以新闻视频为例,给出了基于内容的新闻视频摘要系统(KBVA-1)。

JVT草案中的核心技术综述

作为一种由ITU-T国际视频组织和ISO的MPEG小组联合推出的未来视频处理标准,JVT(joint video team)的主要功能是改进当前压缩标准,通过采用切图技术、高分辨率的亚像素技术以及不同尺寸的块和形状与使用多个参考图进行Intra/Inter预测编码等技术来降低码率,提高压缩质量和压缩效率.同时在网络层通过对信息的封装与优先级控制等技术来实现数据在不同网络之间的透明传输与网络友好.重点介绍了JVT草案中的核心技术,包括JVT体系结构、层次模型以及与VCL(video coding layer)和NAL(network abstraction layer)两层相关具有代表性的关键技术,分析和探讨了当前JVT存在的问题和进一步的研究方向.

监控视频的摘要提取方法研究

分析了监控视频的特点,提出了一种基于帧运动量计算和运动片段分割的监控视频摘要提取方法。实验结果表明,该方法具有快速、准确的特点,具有较高的实用价值。

典型的视频摘要系统及其对比

视频摘要是以自动或半自动的方式对视频的结构和内容进行分析 ,从原视频中提取出有意义的部分 ,并将它们以某种方式进行组合 ,形成简洁的能够充分表现视频语义内容的视频概要。重点介绍了几个典型的视频摘要系统及其涉及到的一些关键技术 ,并对这些典型的视频摘要系统进行对比 ,最后对视频摘要技术进行总结与展望。