PID-Based Bit Allocation Strategy for Asymmetric Stereoscopic Video Coding with Fuzzy Quality Controller

Asymmetric stereoscopic video coding can take advantage of binocular suppression in human vision by representing one of the two views in lower quality.This paper proposes a bit allocation strategy for asymmetric stereoscopic video coding.In order to improve the accuracy of bit allocation and rate control in the left view,a proportionalintegral-derivative controller is adopted.Meanwhile,to control the quality fluctuation between consecutive frames of the left view,a quality controller is adopted.Besides,a fuzzy controller is proposed to control the variation in quality between the left and right views by comparing the PSNR disparity of two views with a fixed threshold,which is used to quantize the binocular psycho-visual redundancy and adjust the quantization parameter (QP) of the right view correspondingly.The proposed algorithm has been implemented in H.264/AVC video codec,and the experimental results show its effectiveness in rate control while keeping a good quality for the left view,and fewer bits are allocated for the right view so that the overall bit rate is saved by 7.2% at most without the loss of subjective visual quality for stereoscopic video.

Interactive transport of multi-view videos for 3DTV applications

The authors propose a novel method for transporting multi-view videos that aims to keep the bandwidth requirements on both end-users and servers as low as possible. The method is based on application layer multicast, where each end point re- ceives only a selected number of views required for rendering video from its current viewpoint at any given time. The set of selected videos changes in real time as the user’s viewpoint changes because of head or eye movements. Techniques for reducing the black-outs during fast viewpoint changes were investigated. The performance of the approach was studied through network experiments.

Multi-view video color correction using dynamic programming

Color inconsistency between views is an important problem to be solved in multi-view video systems. A multi-view video color correction method using dynamic programming is proposed. Three-dimensional histograms are constructed with sequential conditional probability in HSI color space. Then, dynamic programming is used to seek the best color mapping relation with the minimum cost path between target image histogram and source image histogram. Finally, video tracking technique is performed to correct multi-view video. Experimental results show that the proposed method can obtain better subjective and objective performance in color correction.

Color compensation for multi-view video coding based on diversity of cameras

A novel color compensation method for multi-view video coding (MVC) is proposed, which efficiently exploits the inter-view dependencies between views with the existence of color mismatch caused by the diversity of cameras. A color compensation model is developed in RGB channels and then extended to YCbCr channels for practical use. A modified inter-view reference picture is constructed based on the color compensation model, which is more similar to the coding picture than the original inter-view reference picture. Moreover, the color compensation factors can be derived in both encoder and decoder, therefore no additional data need to be transmitted to the decoder. The experimental results show that the proposed method improves the coding efficiency of MVC and maintains good subjective quality.

Frame-layer bit allocation for multi-view video coding based on frame complexity estimation

Current multi-view video coding （MVC） reference model in joint video team （JVT） does not provide efficient rate control schemes. This paper presents a rate control algorithm for MVC by improving the quadratic rate-distortion （R-D） model. We reasonably allocate bit-rate among views based on the correlation analysisl The proposed algorithm consists of three levels to control the rate bits more accurately, of which the frame layer allocates bits according to the frame complexity and the temporal activity. Extensive experiments show that the proposed algorithm can control the bit rate efficiently.

Efficient fast mode decision using mode complexity for multi-view video coding

The variable block-size motion estimation(ME) and disparity estimation(DE) are adopted in multi-view video coding(MVC) to achieve high coding efficiency. However, much higher computational complexity is also introduced in coding system, which hinders practical application of MVC. An efficient fast mode decision method using mode complexity is proposed to reduce the computational complexity. In the proposed method, mode complexity is firstly computed by using the spatial, temporal and inter-view correlation between the current macroblock(MB) and its neighboring MBs. Based on the observation that direct mode is highly possible to be the optimal mode, mode complexity is always checked in advance whether it is below a predefined threshold for providing an efficient early termination opportunity. If this early termination condition is not met, three mode types for the MBs are classified according to the value of mode complexity, i.e., simple mode, medium mode and complex mode, to speed up the encoding process by reducing the number of the variable block modes required to be checked. Furthermore, for simple and medium mode region, the rate distortion(RD) cost of mode 16×16 in the temporal prediction direction is compared with that of the disparity prediction direction, to determine in advance whether the optimal prediction direction is in the temporal prediction direction or not, for skipping unnecessary disparity estimation. Experimental results show that the proposed method is able to significantly reduce the computational load by 78.79% and the total bit rate by 0.07% on average, while only incurring a negligible loss of PSNR(about 0.04 d B on average), compared with the full mode decision(FMD) in the reference software of MVC.

CODING-ORIENTED MULTI-VIEW VIDEO COLOR CORRECTION

Color inconsistency between views is an important problem to be solved in multi-view video applications, such as free viewpoint television and other three-dimensional video systems. In this paper, by combining with multi-view video coding, a coding-oriented multi-view video color correction method is proposed. We first separate foreground and background in first Group Of Pictures （GOP） by using SKIP coding mode. Then by transferring means and standard deviations in backgrounds, color correction is performed for each frame in GOP, and multi-view video coding is performed and used to renew the backgrounds. Experimental results ances in color correction and multi-view video show the proposed method can obtain better performcoding.

Trifocal tensor based side information generation for multi-view distributed video code

Distributed video coding （DVC） is a new video coding approach based on Wyner-Ziv theorem. The novel uplink-friendly DVC, which offers low-complexity, low-power consuming, and low-cost video encoding, has aroused more and more research interests. In this paper a new method based on multiple view geometry is presented for spatial side information generation of uncalibrated video sensor network. Trifocal tensor encapsulates all the geometric relations among three views that are independent of scene structure; it can be computed from image correspondences alone without requiring knowledge of the motion or calibration. Simulation results show that trifocal tensor-based spatial side information improves the rate-distortion performance over motion compensation based interpolation side information by a maximum gap of around 2dB. Then fusion merges the different side information （temporal and spatial） in order to improve the quality of the final one. Simulation results show that the rate-distortion gains about 0.4 dB.

Bandwidth-Efficient Transmission Method for User View-Oriented Video Services

The trend in video viewing has been evolving beyond simply providing a multi-view option.Recently,a function that allows selection and viewing of a clip from a multi-view service that captures a specific range or object has been added.In particular,the free-view service is an extended concept of multi-view and provides a freer viewpoint.However,since numerous videos and additional data are required for its construction,all of the clips constituting the content cannot be simultaneously provided.Only certain clips are selected and provided to the user.If the video is not the preferred video,change request is made,and a delay occurs during retransmission from the server.Delays due to frequent re-requests degrade the overall quality of service.For free-view services,selectively transmitting the video according to the user’s desired viewpoint and region of interest within the limited network of available videos is important.In this study,we propose a method of screening and providing the correct video based on objects in the contents.Based on the method of recognizing the object in each clip,we designed a method of setting its priority based on information about the object’s location for each viewpoint.During the transmission and receiving process using this information,the selected video can be rapidly recognized and changed.Herein,we present a service system configuration method and propose video selection examples for free-view services.

虚拟现实中视觉诱发晕动症时空多特征评价

沉浸式的虚拟现实体验中视觉诱发晕动症(Visually Induced Motion Sickness,VIMS)是影响虚拟现实系统发展与应用的一个重要问题。现有的基于视觉内容的评价方案大多考虑的要素不够全面,对运动信息的提取较为简单,且少有考虑时域上的突变对晕动症的影响。针对上述问题,提出了虚拟现实中视觉诱发晕动症时空多特征评价模型;基于立体全景视频中空域、时域信息来设计视觉诱发晕动症评价模型,采用更符合人眼感知的加权运动特征,并考虑了立体全景视频时域的突变信息设计了特征提取方式。所提出模型分为预处理模块、特征提取模块及时域聚合与回归模块。预处理模块用于视口提取和光流图、视差图、显著图的估计。特征提取模块包含前背景加权运动特征提取、基于变换域的视差特征提取、空间特征提取及时域突变特征提取。时域聚合后通过支持向量回归得到VIMS评价分数。实验结果表明,该模型在立体全景视频数据库SPVCD上的预测结果与平均主观意见分的皮尔逊线性相关系数为0.821、斯皮尔曼相关系数为0.790、均方根误差为0.489。该模型取得了优良的预测性能,验证了所提出特征提取模块的有效性。

A retargeting method for stereoscopic 3D video

We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.

Comfort-driven disparity adjustment for stereoscopic video

Pixel disparity—the offset of corresponding pixels between left and right views—is a crucial parameter in stereoscopic three-dimensional(S3D)video, as it determines the depth perceived by the human visual system(HVS). Unsuitable pixel disparity distribution throughout an S3 D video may lead to visual discomfort. We present a unified and extensible stereoscopic video disparity adjustment framework which improves the viewing experience for an S3 D video by keeping the perceived 3D appearance as unchanged as possible while minimizing discomfort. We first analyse disparity and motion attributes of S3 D video in general, then derive a wide-ranging visual discomfort metric from existing perceptual comfort models. An objective function based on this metric is used as the basis of a hierarchical optimisation method to find a disparity mapping function for each input video frame. Warping-based disparity manipulation is then applied to the input video to generate the output video, using the desired disparity mappings as constraints. Our comfort metric takes into account disparity range, motion, and stereoscopic window violation; the framework could easily be extended to use further visual comfort models. We demonstrate the power of our approach using both animated cartoons and real S3 D videos.

Extraction technique of region of interest from stereoscopic video

A feature fusion approach is presented to extract the region of interest （ROI） from the stereoscopic video. Based on human vision system （HVS）, the depth feature, the color feature and the motion feature are chosen as vision features. The algorithm is shown as follows. Firstly, color saliency is calculated on superpixel scale. Color space distribution of the superpixel and the color difference between the superpixel and background pixel are used to describe color saliency and color salient region is detected. Then, the classic visual background extractor （Vibe） algorithm is improved from the update interval and update region of background model. The update interval is adjusted according to the image content. The update region is determined through non-obvious movement region and background point detection. So the motion region of stereoscopic video is extracted using improved Vibe algorithm. The depth salient region is detected by selecting the region with the highest gray value. Finally, three regions are fused into final ROI. Experiment results show that the proposed method can extract ROI from stereoscopic video effectively. In order to further verify the proposed method, stereoscopic video coding application is also carried out on the joint model （JM） encoder with different bit allocation in RO｜ and the background region.

视频立体化健康教育结合心理护理对CT检查患者心理状态及影像质量的影响

目的:探讨视频立体化健康教育结合心理护理对CT检查患者心理状态及影像质量的影响.方法:选取2021年1月至2023年6月于本院进行CT检查的106例患者为研究对象.根据护理方案将其分为观察组(n=56,采用视频立体化健康教育结合心理护理干预)和对照组(n=50,采用常规护理干预).采用统计学方法,比较两组患者影像质量、检查配合度以及心理状态[采用焦虑自评量表(SAS)和抑郁自评量表(SDS)评估].结果:采取不同干预措施后,观察组患者整体图像质量高于对照组,差异有统计学意义(P<0.05);观察组患者整体配合度高于对照组,差异有统计学意义(P<0.05);两组患者干预后SAS、SDS评分均下降,并且观察组低于对照组,差异均有统计学意义(P<0.05).结论:视频立体化健康教育结合心理护理可提高CT检查患者图像质量与检查配合度,有效改善心理状态,具有临床推广与应用价值.

三维视频编码技术的发展与挑战

介绍了三维视频编码的核心技术。首先比较了纯视频格式和深度增强格式三维视频编码技术的发展方向及面临的挑战,其中重点分析了深度估计技术和视点合成技术的研究思路;然后概括了国际3DV/FTV标准的制定现状;最后对三维视频编码技术进行了总结和展望。

立体图像视频格式及其转换技术研究

在视差原理的基础上,结合双目立体视觉几何模型,得出裸眼立体显示的光路分布特点,并在此基础上研制成功立体显示器样机。对立体视图的映射关系做了详细解析,提出了合适的立体图像视频格式及其转换算法,实验结果表明研究方法和结论正确。

亮度优化立体视频视觉舒适度评价

结合立体视频帧左右视图的灰度图,采用鲁棒性算法与软剪刀算法相结合的"画笔"方案来提取立体视频帧的前景区域,合成视差图并提取水平视差,计算深度视差;根据主观评价结果建立视差深度—视觉舒适度模型,再结合亮度对视觉舒适度的评价模型进行优化,使模型的评分更接近人类视觉系统的评分;结合人类主观评价实验的结果,用逆向代入法求出模型的系数确定模型,最终实现基于人类视觉系统特性的对立体视频视觉舒适度的评价。实验结果表明:通过视觉舒适度模型计算得到的结果与人类主观评价结果之间的误差率不超过5%,其中大部分视频误差率不超过1%。说明评价模型更符合人类主观感知结果的立体视频视觉舒适度,模型评分更接近人类视觉系统特性的主观实验评分,这为立体视频舒适度的评价提供了参考。

基于裸眼立体显示的视频变换技术研究

裸眼立体显示图像和视频等数据源,要满足规定的格式才可以与特定的立体显示器相匹配.为了兼容传统的立体显示数据源格式,并立足于裸眼立体显示的要求,就新研制的裸眼液晶立体显示器提出了对数据格式的要求及其转换算法.

面向立体视频的视差-运动同步联立预测算法

针对立体视频序列的时间空间冗余和交叉冗余,提出一种快速编码算法.应用立体-运动约束模型在视差域和运动域进行联合迭代搜索,根据每次迭代后的模型误差设置自适应修正窗口来保证预测矢量精度,视差矢量和运动矢量可以快速同步获得.实验结果表明,该算法在保证高率失真性能的同时,可以大幅度降低编码复杂度.

基于H.264的立体视频编码方法

H.264是一种高效的视频编码压缩标准,它集中了以往标准的优点。基于H.264的高效编码压缩技术,文章研究了块基立体视频编码方法,并对基于H.264和基于H.263+的立体视频编码方案进行了编码效率的实验比较。实验结果表明,基于H.264的编码方案大大超过了基于H.263+的编码方案,是一种高效的立体视频编码压缩方案。