Soft transmission of 3D video for low power and low complexity scenario

Thanks to the rapid development of naked-eye 3D and wireless communication technology,3D video related applications on mobile devices have attracted a lot of attention.Nevertheless,the time-varying characteristics of the wireless channel is very challenging for conventional source-channel coding based transmission strategy.Also,the high complexity of source-channel coding based transmission scheme is undesired for low power mobile terminals.An advanced transmission scheme named Softcast was proposed to achieve efficient transmission performance for 2D image/video.Unfortunately,it cannot be directly applied to wireless 3D video transmission with high efficiency.This paper proposes a more efficient soft transmission scheme for 3D video with a graceful quality adaptation within a wide range of channel Signal-to-Noise Ratio(SNR).The proposed method first extends the linear transform to 4 dimensions with additional view dimension to eliminate the view redundancy,and then metadata optimization and chunk interleaving are designed to further improve the transmission performance.Meanwhile,a synthesis distortion based chunk discard strategy is developed to improve the overall 3D video quality under the condition of limited bandwidth.The experimental results demonstrate that the proposed method significantly improves the 3D video transmission performance over the wireless channel for low power and low complexity scenarios.

A VIDEO SPECTRUM SPLITTING ENCODING SCHEME BASED ON HUMAN VISION AND ITS COMPUTER SIMULATION

In this paper, a 3-D video encoding scheme suitable for digital TV/HDTV (high definition television) is studied through computer simulation. The encoding scheme is designed to provide a good match to human vision. Basically, this involves transmission of low frequency luminance information at full frame rate for good motion rendition and transmission of high frequency luminance signal at reduced frame rate for good detail in static images.

Web-based interactive visualization of 3D video mosaics using X3D standard

We present a method of 3D image mosaicing for real 3D representation of roadside buildings, and implement a Web-based interactive visualization environment for the 3D video mosaics created by 3D image mosaicing. The 3D image mo- saicing technique developed in our previous work is a very powerful method for creating textured 3D-GIS data without excessive data processing like the laser or stereo system. For the Web-based open access to the 3D video mosaics, we build an interactive visualization environment using X3D, the emerging standard of Web 3D. We conduct the data preprocessing for 3D video mosaics and the X3D modeling for textured 3D data. The data preprocessing includes the conversion of each frame of 3D video mosaics into concatenated image files that can be hyperlinked on the Web. The X3D modeling handles the representation of concatenated images using necessary X3D nodes. By employing X3D as the data format for 3D image mosaics, the real 3D representation of roadside buildings is extended to the Web and mobile service systems.

AVS 3D Video Coding Technology and System

Following the success of the audio video standard （AVS） for 2D video coding, in 2008, the China AVS workgroup started developing 3D video （3DV） coding techniques. In this paper, we discuss the background, technical features, and applications of AVS 3DV coding technology. We introduce two core techniques used in AVS 3DV coding： inter-view prediction and enhanced stereo packing coding. We elaborate on these techniques, which are used in the AVS real-time 3DV encoder. An application of the AVS 3DV coding system is presented to show the great practical value of this system. Simulation results show that the advanced techniques used in AVS 3DV coding provide remarkable coding gain compared with techniques used in a simulcast scheme.

A Distributed 2D-to-3D Video Conversion System

2D-to-3D video conversion is a feasible way to generate 3D programs for the current 3DTV industry. However, for large-scale 3D video production, current systems are no longer adequate in terms of the time and labor required for conversion. In this paper, we introduce a distributed 2D-to-3D video conversion system that includes a 2D-to-3D video conversion module, architecture of the parallel computation on the cloud, and 3D video coding in the system. The system enables cooperation among multiple users in the simultaneous completion of their conversion tasks so that the conversion efficiency is greatly promoted. In the experiments, we evaluate the system based on criteria related to both time consumption and video coding performance.

3D Perception Algorithms: Towards Perceptually Driven Compression of 3D Video

In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D video viewing. 3D video requires depth perception, which implies binocular effects such as con fl icts, fusion, and rivalry. A better understanding of these effects is necessary for 3D perceptual compression, which provides users with a more comfortable visual experience for video that is de- livered over a channel with limited bandwidth. We present state-of-the-art of 3D visual attention models, 3D just-notice- able difference models, and 3D texture-synthesis models that address 3D human vision issues in 3D video coding and trans-mission.

Dynamic Hyperlinker: Innovative Solution for 3D Video Content Search and Retrieval

Recently, 3D display technology, and content creation tools have been undergone rigorous development and as a result they have been widely adopted by home and professional users. 3D digital repositories are increasing and becoming available ubiquitously. However, searching and visualizing 3D content remains a great challenge. In this paper, we propose and present the development of a novel approach for creating hypervideos, which ease the 3D content search and retrieval. It is called the dynamic hyperlinker for 3D content search and retrieval process. It advances 3D multimedia navigability and searchability by creating dynamic links for selectable and clickable objects in the video scene whilst the user consumes the 3D video clip. The proposed system involves 3D video processing, such as detecting/tracking clickable objects, annotating objects, and metadata engineering including 3D content descriptive protocol. Such system attracts the attention from both home and professional users and more specifically broadcasters and digital content providers. The experiment is conducted on full parallax holoscopic 3D videos “also known as integral images”.

A simplified hardware-friendly contour prediction algorithm in 3D-HEVC and parallelization design

After the extension of depth modeling mode 4(DMM-4)in 3D high efficiency video coding(3D-HEVC),the computational complexity increases sharply,which causes the real-time performance of video coding to be impacted.To reduce the computational complexity of DMM-4,a simplified hardware-friendly contour prediction algorithm is proposed in this paper.Based on the similarity between texture and depth map,the proposed algorithm directly codes depth blocks to calculate edge regions to reduce the number of reference blocks.Through the verification of the test sequence on HTM16.1,the proposed algorithm coding time is reduced by 9.42%compared with the original algorithm.To avoid the time consuming of serial coding on HTM,a parallelization design of the proposed algorithm based on reconfigurable array processor(DPR-CODEC)is proposed.The parallelization design reduces the storage access time,configuration time and saves the storage cost.Verified with the Xilinx Virtex 6 FPGA,experimental results show that parallelization design is capable of processing HD 1080p at a speed above 30 frames per second.Compared with the related work,the scheme reduces the LUTs by 42.3%,the REG by 85.5%and the hardware resources by 66.7%.The data loading speedup ratio of parallel scheme can reach 3.4539.On average,the different sized templates serial/parallel speedup ratio of encoding time can reach 2.446.

Recent Advances and Challenges of Visual Signal Quality Assessment

While quality assessment is essential for testing, optimizing, benchmarking, monitoring, and inspecting related systems and services, it also plays an essential role in the design of virtually all visual signal processing and communication algorithms, as well as various related decision-making processes. In this paper, we first provide an overview of recently derived quality assessment approaches for traditional visual signals (i.e., 2D images/videos), with highlights for new trends (such as machine learning approaches). On the other hand, with the ongoing development of devices and multimedia services, newly emerged visual signals (e.g., mobile/3D videos) are becoming more and more popular. This work focuses on recent progresses of quality metrics, which have been reviewed for the newly emerged forms of visual signals, which include scalable and mobile videos, High Dynamic Range (HDR) images, image segmentation results, 3D images/videos, and retargeted images.

Depth Enhancement Methods for Centralized Texture-Depth Packing Formats

To deliver three-dimension （3D） videos through the current two-dimension （2D） broadcasting systems, the frame-compati-ble packing formats properly including one texture frame and one depth map in various down-sampling ratios have been proposed to achieve the simplest and most effective solution. To enhance the compatible centralized texture-depth packing （CTDP） formats, in this paper, we further introduce two depth enhancement algorithms to further improve the quality of CT-DP formats for delivering 3D video services. To compensate the loss of color YCbCr 444 to 420 conversion of colored-depth, two efficient depth reconstruction processes based on texture and depth consistency are proposed. Experimental re-sults show that the proposed enhanced CTDP depacking pro-cess outperforms the 2DDP format and the original CTDP de-packing procedure in synthesizing virtual views. With the help of the proposed efficient depth reconstruction processes, more correct reconstructed depth maps and better synthesized quality can be achieved. Before the available 3D broadcasting systems, which adopt truly depth and texture dependent cod-ing procedure, we believe that the proposed CTDP formats with depth enhancement could help to deliver 3D videos in the current 2D broadcasting systems simply and efficiently.

3DV quality model based depth maps for view synthesis in FTV system

Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts in the synthesized views. To solve this problem, a 3D video quality model base depth maps （D-3DV） for virtual view synthesis and depth map coding in the FTV applications is proposed. First, the relationships between distortions in coded depth map and rendered view are derived. Then, a precisely 3DV quality model based depth characteristics is develop for the synthesized virtual views. Finally, based on D-3DV model, a multilateral filtering is applied as a pre-processed filter to reduce rendering artifacts. The experimental results evaluated by objective and subjective methods indicate that the proposed D-3DV model can reduce bit-rate of depth coding and achieve better rendering quality.

Wedge template optimization and parallelization of depth map in intra-frame prediction algorithms

To reduce the computational complexity and storage cost caused by wedge segmentation algorithm,a scheme of simplifying wedge matching is proposed.It takes advantage of the correlation of the wedge separation line of depth map and the direction of intra-prediction for 3D high-efficiency video coding(3D-HEVC).According to the difference of wedge segmentation between adjacent edge and opposite edge,a set only including 104×4 wedgelet templates is given.By expanding of the wedge wave of a certain minimum unit,a simple separation line acquisition method for different size of depth block is put forward.Furthermore,based on the array processor(DPR-CODEC)developed by project team,an efficient parallel scheme of the improved wedge segmentation mode prediction is introduced.By the scheme,prediction unit(PU)size can be changed randomly from 4×4 to 8×8,16×16,and 32×32,which is more in line with the needs of the HEVC standard.Veri-fied with test sequence in HTM16.1 and the Xilinx virtex-6 field programmable gate array(FPGA)respectively,the experiment results show that the proposed methods save 99.2%of the storage space and 63.94%of the encoding time,the serial/parallel acceleration ratio of each template reaches 1.84 in average.The coding performance,storage and resource consumption are considered for both.

3D displays: toward holographic video displays of 3D images

As the fiat panel displays （Liquid Crystal Displays, AMOLED, etc.） reach near perfection in their viewing qualities and display areas, it is natural to seek the next level of displays, including 3D displays. There is a strong surge in 3D liquid crystal displays as a result of the successful movie Avatar. Most of these 3D displays involve the employment of special glasses that allow one view perspective for each of the eyes to achieve a depth perception. Such displays are not real 3D displays. In fact, these displays can only provide one viewing perspective for all viewers, regardless of the viewer＇s position. In addition, a fundamental viewing problem of focusing and accommodation exist that can lead to discomfort and fatigue for many viewers. In this paper, the authors review the current status of stereoscopic 3D displays and their problems. The authors will also discuss the possibility of using fiat panels for the display of both phase and intensity of video image information, leading to the ultimate display of 3D holographic video images. Many of the fundamental issues and limitations will be presented and discussed.

A retargeting method for stereoscopic 3D video

We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.

Computational approaches for fast generation of digital 3D video holograms(Invited Paper)

Several approaches for fast generation of digital holograms of a three-dimensional （3D） object have been discussed. Among them, the novel look-up table （N-LUT） method is analyzed to dramatically reduce the number of pre-calculated fringe patterns required for computation of digital holograms of a 3D object by employing a new concept of principal fringe patterns, so that problems of computational complexity and huge memory size of the conventional ray-tracing and look-up table methods have been considerably alleviated. Meanwhile, as the 3D video images have a lot of temporally or spatially redundant data in their inter- and intra-frames, computation time of the 3D video holograms could be also reduced just by removing these redundant data. Thus, a couple of computational methods for generation of 3D video holograms by combined use of the N-LUT method and data compression algorithms are also presented and discussed. Some experimental results finally reveal that by using this approach a great reduction of computation time of 3D video holograms could be achieved.