Color compensation for multi-view video coding based on diversity of cameras

A novel color compensation method for multi-view video coding (MVC) is proposed, which efficiently exploits the inter-view dependencies between views with the existence of color mismatch caused by the diversity of cameras. A color compensation model is developed in RGB channels and then extended to YCbCr channels for practical use. A modified inter-view reference picture is constructed based on the color compensation model, which is more similar to the coding picture than the original inter-view reference picture. Moreover, the color compensation factors can be derived in both encoder and decoder, therefore no additional data need to be transmitted to the decoder. The experimental results show that the proposed method improves the coding efficiency of MVC and maintains good subjective quality.

Frame-layer bit allocation for multi-view video coding based on frame complexity estimation

Current multi-view video coding （MVC） reference model in joint video team （JVT） does not provide efficient rate control schemes. This paper presents a rate control algorithm for MVC by improving the quadratic rate-distortion （R-D） model. We reasonably allocate bit-rate among views based on the correlation analysisl The proposed algorithm consists of three levels to control the rate bits more accurately, of which the frame layer allocates bits according to the frame complexity and the temporal activity. Extensive experiments show that the proposed algorithm can control the bit rate efficiently.

Efficient fast mode decision using mode complexity for multi-view video coding

The variable block-size motion estimation(ME) and disparity estimation(DE) are adopted in multi-view video coding(MVC) to achieve high coding efficiency. However, much higher computational complexity is also introduced in coding system, which hinders practical application of MVC. An efficient fast mode decision method using mode complexity is proposed to reduce the computational complexity. In the proposed method, mode complexity is firstly computed by using the spatial, temporal and inter-view correlation between the current macroblock(MB) and its neighboring MBs. Based on the observation that direct mode is highly possible to be the optimal mode, mode complexity is always checked in advance whether it is below a predefined threshold for providing an efficient early termination opportunity. If this early termination condition is not met, three mode types for the MBs are classified according to the value of mode complexity, i.e., simple mode, medium mode and complex mode, to speed up the encoding process by reducing the number of the variable block modes required to be checked. Furthermore, for simple and medium mode region, the rate distortion(RD) cost of mode 16×16 in the temporal prediction direction is compared with that of the disparity prediction direction, to determine in advance whether the optimal prediction direction is in the temporal prediction direction or not, for skipping unnecessary disparity estimation. Experimental results show that the proposed method is able to significantly reduce the computational load by 78.79% and the total bit rate by 0.07% on average, while only incurring a negligible loss of PSNR(about 0.04 d B on average), compared with the full mode decision(FMD) in the reference software of MVC.

CODING-ORIENTED MULTI-VIEW VIDEO COLOR CORRECTION

Color inconsistency between views is an important problem to be solved in multi-view video applications, such as free viewpoint television and other three-dimensional video systems. In this paper, by combining with multi-view video coding, a coding-oriented multi-view video color correction method is proposed. We first separate foreground and background in first Group Of Pictures （GOP） by using SKIP coding mode. Then by transferring means and standard deviations in backgrounds, color correction is performed for each frame in GOP, and multi-view video coding is performed and used to renew the backgrounds. Experimental results ances in color correction and multi-view video show the proposed method can obtain better performcoding.

Trifocal tensor based side information generation for multi-view distributed video code

Distributed video coding （DVC） is a new video coding approach based on Wyner-Ziv theorem. The novel uplink-friendly DVC, which offers low-complexity, low-power consuming, and low-cost video encoding, has aroused more and more research interests. In this paper a new method based on multiple view geometry is presented for spatial side information generation of uncalibrated video sensor network. Trifocal tensor encapsulates all the geometric relations among three views that are independent of scene structure; it can be computed from image correspondences alone without requiring knowledge of the motion or calibration. Simulation results show that trifocal tensor-based spatial side information improves the rate-distortion performance over motion compensation based interpolation side information by a maximum gap of around 2dB. Then fusion merges the different side information （temporal and spatial） in order to improve the quality of the final one. Simulation results show that the rate-distortion gains about 0.4 dB.

基于HBP编码结构的MVC快速算法

针对MVC系统中运算量最大的矢量预测部分——单向搜索和双向搜索,该文提出一种快速算法。根据多视点视频的序列相关性,采用立体-运动约束模型在运动域和视差域进行联合迭代单向搜索,设计可信度因子来自适应地调整修正搜索窗口,同步搜索得到运动和视差矢量。此外,通过分析双向预测宏块的特性,提出一种双向搜索提前中止算法。实验结果证明该文算法的有效性,能在保持编码质量的同时极大减少算法复杂度。

基于SVM的MVC快速宏块模式选择算法

JMVC编码器采用全搜索算法在视点间和时间参考帧中遍历所有可能的预测模式选择最优,以获得最佳编码质量,在获得高压缩比的同时也带来非常高的编码复杂度,严重影响多视点视频的实际应用.针对此问题,利用JMVC多视点视频编码模型提出一种基于SVM的多视点视频编码快速宏块模式选择算法,利用SVM方法训练参考视点中提取到的相关信息,进行分级3层的SVM归类.对采用的3层分类策略进行讨论,并对特征参数的选取进行详细描述.实验结果表明,与JMVC全搜索算法相比,本文算法可在保持视频编码质量的同时节省大量编码时间.

多阵列摄像图像调节的MVC颜色补偿方法

多视点视频是由一组摄像机拍摄得到的视频序列。通常情况下,各个摄像机的性能参数很难做到完全一致,从而引起不同视点的视频图像之间产生颜色差异。提出了在多视点视频编码中使用一种新的颜色补偿模式,这种模式能够有效利用相互独立编码的帧间视频的信息,从而减少由多阵列摄像而带来的颜色误匹配。该方法在RGB颜色空间上建立颜色补偿模型,并进一步将颜色补偿模型扩展到YCbCr颜色空间以适用现有视频编码标准。与原始视点间参考图像相比,经过补偿后的视点间参考图像与当前编码图像更为相似,从而提高了MVC的编码效率。此外,颜色补偿模型参数可以分别从编码端和解码端推导得出,不需要将其写入码流。实验结果也表明提出的方法能够提高编码的效率和得到更好的视频质量。

基于相关性分析的MVC比特分配算法

针对目前尚未深入研究多视点视频编码(Multi-view Video Coding,MVC)码率控制的问题,提出了一种基于相关性分析的多视点视频编码码率控制算法。该算法的核心是先根据视差预测和运动预测的结构关系,将所有图像分成6种类型的编码帧,并改进二项式率失真模型,然后根据多视点视频相关性分析在各个视点之间进行合理的码率分配,将码率控制分成4层结构进行多视点视频编码的码率控制。其中,帧层码率控制考虑分层B帧等因素分配码率,基本单元层码率控制根据宏块的内容复杂度采用不同的量化参数。实验结果表明该码率控制算法实际码率与目标码率平均误差能控制0.6%。

Bandwidth-Efficient Transmission Method for User View-Oriented Video Services

The trend in video viewing has been evolving beyond simply providing a multi-view option.Recently,a function that allows selection and viewing of a clip from a multi-view service that captures a specific range or object has been added.In particular,the free-view service is an extended concept of multi-view and provides a freer viewpoint.However,since numerous videos and additional data are required for its construction,all of the clips constituting the content cannot be simultaneously provided.Only certain clips are selected and provided to the user.If the video is not the preferred video,change request is made,and a delay occurs during retransmission from the server.Delays due to frequent re-requests degrade the overall quality of service.For free-view services,selectively transmitting the video according to the user’s desired viewpoint and region of interest within the limited network of available videos is important.In this study,we propose a method of screening and providing the correct video based on objects in the contents.Based on the method of recognizing the object in each clip,we designed a method of setting its priority based on information about the object’s location for each viewpoint.During the transmission and receiving process using this information,the selected video can be rapidly recognized and changed.Herein,we present a service system configuration method and propose video selection examples for free-view services.

Local curve fitting based Lagrange multiplier selection for Id-slice in multi-view video coding

The rate and distortion of Id-slice do not fit the globally linear relationship on a logarithmic scale. Lagrange multiplier selection methods based on the globally linear approximate relationship are neither efficient nor optimal for multi-view video coding (MVC). To improve the coding efficiency of MVC, a local curve fitting based Lagrange multiplier selection method is proposed in this paper, where Lagrange multipliers are selected according to the local slopes of the approximate curves. Experi-mental results showed that the proposed method improves the coding efficiency. Up to 2.5 dB gain was achieved at low bitrates.

Adaptive data-driven parallelization of multi-view video coding on multi-core processor

Multi-view video coding （MVC） comprises rich 3D information and is widely used in new visual media, such as 3DTV and free viewpoint TV （FTV）. However, even with mainstream computer manufacturers migrating to multi-core processors, the huge computational requirement of MVC currently prohibits its wide use in consumer markets. In this paper, we demonstrate the design and implementation of the first parallel MVC system on Cell Broadband Engine^TM processor which is a state-of-the-art multi-core processor. We propose a task-dispatching algorithm which is adaptive data-driven on the frame level for MVC, and implement a parallel multi-view video decoder with modified H.264/AVC codec on real machine. This approach provides scalable speedup （up to 16 times on sixteen cores） through proper local store management, utilization of code locality and SIMD improvement. Decoding speed, speedup and utilization rate of cores are expressed in experimental results.

Optimizing inter-view prediction structures for multi-view video coding using simulated annealing

New video applications, such as 3D video and free viewpoint video, require efficient compression of multi-view video. In addition to temporal redundancy, exploiting the inter-view redundancy is crucial to improve the performance of multi-view video coding. In this paper, we present a novel method to construct the optimal inter-view prediction structure for multi-view video coding using simulated annealing. In the proposed model, the design of the prediction structure is converted to the arrangement of coding order. Then, a simulated annealing algorithm is employed to minimize the total cost for obtaining the best coding order. This method is applicable to arbitrary irregular camera arrangements. As experiment results reveal, the annealing process converges to satisfactory results rapidly and the generated optimal prediction structure outperforms the reference prediction structure of the joint multi-view video model (JMVM) by 0.1-0.8 dB PSNR gains.

FTV系统中面向虚拟视点合成的深度编码

为了降低自由视点电视系统中由于深度估计不准确和深度数据压缩带来的量化失真对重建虚拟视点图像质量产生的影响,提出了一种面向虚拟视点绘制的深度压缩算法.根据深度数据的统计特性,设计了一种基于三次B样条函数的上下采样方法对深度图进行下采样.采用多视点视频编码对下采样深度图进行编码,在解码端则对深度图进行上采样,并针对解码深度图边缘处的模糊现象,用双边滤波器优化深度图边界.实验结果表明,与传统的MVC编码方案相比,该方案在保持虚拟绘制图像质量不变(甚至略有提升)的情况下,显著降低了传输码率,有助于在自由视点系统的接收端获得高精度的重建图像.

一种新的视点间预测结构

多视点视频编码除应具有较高的编码效率外,还应该包括后向兼容性、时间随机访问和视点可分级性等,这些都主要取决于所采用的预测结构。目前所提供的多视点视频编码(Joint Multi-view Video Coding,JMVC)采用固定的视点间预测结构,难以适应复杂情况的多视点视频编码。该文综合考虑编码效率和用户随机访问等因素,根据多视点视频相关性分析自适应调整视点间预测结构,以获得较好的编码综合性能。试验结果表明,与JMVC相比,该文的方法在提高编码效率的同时,有较好的随机访问性能。

基于多维矢量矩阵的多视角视频编码

针对现行视频压缩编码标准对多视角视频编码(MVC)的支持与扩展存在一定的局限性这一问题,结合多维矢量矩阵理论,对先前研究中提出的针对单一视角彩色视频流的一整套压缩编码方案加以扩展,将其应用于八个视角的视频编码中,包括多维分块、多维重组、多维矢量DCT正交变换、多维量化、差分编码、多维扫描及行程编码。通过对实验数据的分析,验证了这一整套方案在多视角视频编码中的可行性,为多维矢量矩阵以及多视角视频编码的进一步研究奠定了基础。

基于相关性分析的多模式多视点视频编码

多视点视频编码方法除需具有较高编码效率外,还必须支持视点或时间的随机访问、低延时编解码、视点可分级等性能.多视点视频信号的时间、视点间相关性随相机密度、光照、对象运动等因素不同而变化.文中提出基于多视点视频信号相关性分析的多模式多视点视频编码方法,改变传统单一预测模式的多视点编码结构,将多种性能优良的预测编码模式有机结合,根据多视点视频相关性分析灵活选择合适的预测编码模式,以获得优异的编码综合性能.实验结果表明,所提出的多模式多视点视频编码方法在保证高压缩效率的前提下,可进一步降低复杂度,提高随机访问性能.

应用于多视点视频压缩的多参考B帧快速编码算法

基于H.264平台就多视点视频压缩中多参考帧技术存在计算复杂度高的现状,根据多视点视频序列特性,提出了一种针对多参考B帧的快速多视点视频编码压缩算法。新算法改进了多参考帧列表策略,并结合多视点视频序列的时空域相关性特性,提出了适用于多视点视频编码的参考帧列表调整策略。此外,还针对多种块模式编码引入了块模式预判策略。新算法在保证率失真性能的基础上,有效地降低了计算复杂度,提高了多视点视频编码速度。虽然本文是基于M-Picture编码结构实现了新算法,但它同样适用于其它应用多参考B帧预测技术的多视点视频编码结构中。

采用相关性分析的视点可分级编码

为了满足不同用户端需求,在多视点视频编码(multi-view coding,MVC)中融入视点可分级编码。根据多视点视频序列的相关性确定基本视点以及各个增强层视点,并给出了合理的视点间预测的视点可分级视频编码。实验结果表明,所提出的多视点视频编码方法除支持视点可分级外,还提高了信噪比以及随机访问性能。

多视点视频编码快速帧间模式选择算法

为解决多视点视频编码(MVC)可变尺寸块的模式选择计算复杂度过大的问题,提出了基于模式复杂度的多视点视频编码快速模式选择算法。该算法首先分析了多视点视频编码参考模型(JMVC)中各尺寸块的分布特点。然后,提出模式复杂度的概念以确定当前宏块的模式特征。最后,将宏块分成3个不同的模式类型:如果当前宏块属于简单模式,那么仅16×16分块被检查,其他分块均跳过;若当前宏块属于中等模式,则8×8块被跳过;若当前宏块属于复杂模式,所有模式分块都要检查。这样算法对那些不必要的模式选择过程就可以提前终止,从而使得计算量大大减少。实验结果表明:所提算法能保持同JMVC中全搜索算法几乎相同的编码效率,同时使计算复杂度减少62.75%。