Recent Advances in Video Coding for Machines Standard and Technologies

To improve the performance of video compression for machine vision analysis tasks,a video coding for machines(VCM)standard working group was established to promote standardization procedures.In this paper,recent advances in video coding for machine standards are presented and comprehensive introductions to the use cases,requirements,evaluation frameworks and corresponding metrics of the VCM standard are given.Then the existing methods are presented,introducing the existing proposals by category and the research progress of the latest VCM conference.Finally,we give conclusions.

Overview of the Second Generation AVS Video Coding Standard(AVS2)

AVS2 is a new generation video coding standard developed by the AVS working group. Compared with the first generation AVS video coding standard, known as AVS1, AVS2 significantly improves coding performance by using many new coding technologies, e.g., adaptive block partition and two level transform coding. Moreover, for scene video, e.g. surveillance video and conference vid?eo, AVS2 provided a background picture modeling scheme to achieve more accurate prediction, which can also make object detec?tion and tracking in surveillance video coding more flexible. Experimental results show that AVS2 is competitive with High Effi?ciency Video Coding (HEVC) in terms of performance. Especially for scene video, AVS2 can achieve 39% bit rate saving over HEVC.

Cooperative Caching for Scalable Video Coding Using Value-Decomposed Dimensional Networks

Scalable video coding(SVC)has been widely used in video-on-demand(VOD)service,to efficiently satisfy users’different video quality requirements and dynamically adjust video stream to timevariant wireless channels.Under the 5G network structure,we consider a cooperative caching scheme inside each cluster with SVC to economically utilize the limited caching storage.A novel multi-agent deep reinforcement learning(MADRL)framework is proposed to jointly optimize the video access delay and users’satisfaction,where an aggregation node is introduced helping individual agents to achieve global observations and overall system rewards.Moreover,to cope with the large action space caused by the large number of videos and users,a dimension decomposition method is embedded into the neural network in each agent,which greatly reduce the computational complexity and memory cost of the reinforcement learning.Experimental results show that:1)the proposed value-decomposed dimensional network(VDDN)algorithm achieves an obvious performance gain versus the traditional MADRL;2)the proposed VDDN algorithm can handle an extremely large action space and quickly converge with a low computational complexity.

Frame-layer bit allocation for multi-view video coding based on frame complexity estimation

Current multi-view video coding （MVC） reference model in joint video team （JVT） does not provide efficient rate control schemes. This paper presents a rate control algorithm for MVC by improving the quadratic rate-distortion （R-D） model. We reasonably allocate bit-rate among views based on the correlation analysisl The proposed algorithm consists of three levels to control the rate bits more accurately, of which the frame layer allocates bits according to the frame complexity and the temporal activity. Extensive experiments show that the proposed algorithm can control the bit rate efficiently.

A WYNER-ZIV VIDEO CODING METHOD UTILIZING MIXTURE CORRELATION NOISE MODEL

In Wyner-Ziv (WZ) Distributed Video Coding (DVC), correlation noise model is often used to describe the error distribution between WZ frame and the side information. The accuracy of the model can influence the performance of the video coder directly. A mixture correlation noise model in Discrete Cosine Transform (DCT) domain for WZ video coding is established in this paper. Different correlation noise estimation method is used for direct current and alternating current coefficients. Parameter estimation method based on expectation maximization algorithm is used to estimate the Laplace distribution center of direct current frequency band and Mixture Laplace-Uniform Distribution Model (MLUDM) is established for alternating current coefficients. Experimental results suggest that the proposed mixture correlation noise model can describe the heavy tail and sudden change of the noise accurately at high rate and make significant improvement on the coding efficiency compared with the noise model presented by DIStributed COding for Video sERvices (DISCOVER).

AVS 3D Video Coding Technology and System

Following the success of the audio video standard （AVS） for 2D video coding, in 2008, the China AVS workgroup started developing 3D video （3DV） coding techniques. In this paper, we discuss the background, technical features, and applications of AVS 3DV coding technology. We introduce two core techniques used in AVS 3DV coding： inter-view prediction and enhanced stereo packing coding. We elaborate on these techniques, which are used in the AVS real-time 3DV encoder. An application of the AVS 3DV coding system is presented to show the great practical value of this system. Simulation results show that the advanced techniques used in AVS 3DV coding provide remarkable coding gain compared with techniques used in a simulcast scheme.

3-D Wavelet Transform for Very Low Bit-Rate Video Coding

A new improved Goh's 3 D wavelet transform(WT) coding scheme is presented in this paper. The new scheme has great advantages including a simple code structure, low computation cost and good performance in PSNR, compression ratios and visual quality of reconstructions, when compared to the other existing 3 D WT coding methods and the 2 D WT based coding methods. The new 3 D WT coding scheme is suitable for very low bit rate video coding.

Very Low Bit Rate Video Coding Using Motion Compensated 3 D Wavelet Transform

A new motion compensated 3 D wavelet transform (MC 3DWT) video coding scheme is presented in this paper. The new coding scheme has a good performance in average PSNR, compression ratio and visual quality of reconstructions compared with the existing 3 D wavelet transform (3DWT) coding methods and motion compensated 2 D wavelet transform (MC WT) coding method. The new MC 3DWT coding scheme is suitable for very low bit rate video coding.

An Introduction to High Efficiency Video Coding Range Extensions

High Efficiency Video Coding (HEVC) is the latest international video coding standard, which can provide the similar quality with about half bandwidth compared with its predecessor, H.264/MPEG?4 AVC. To meet the requirement of higher bit depth coding and more chroma sampling formats, range extensions of HEVC were developed. This paper introduces the coding tools in HEVC range extensions and provides experimental results to compare HEVC range extensions with previous video coding standards. Ex?perimental results show that HEVC range extensions improve coding efficiency much over H.264/MPEG?4 AVC High Predictive profile, especially for 4K sequences.

Design of the Quantization Matrix in the Distributed Compressed Sensing Video Coding*

In the frame of compressed sensing distributed video coding, the design of the quantization matrix directly affects the reconstruction quality of the receiving terminal of the video. In this article, we present a new design method of the Gaussian quantization matrix adapting to the compressed sensing coding, for that the distribution of the parameters of the image is featured of the characteristic of approximately normal distribution after measured by compressive sensing. By this way, the parameters of a certain quantity of the image frames depending on the video sequences generated by the Gaussian quantization matrix possess certain adaptive capacity. By comparison with the plan of the traditional quantization, the quantization matrix presented in this article would improve the reconstruction quality of the video.

Enhanced Adaptive Approach of Video Coding at Very Low Bit Rate Using MSPIHT Algorithm

Nowadays video coding approach is a major key in many applications for easy transmission and storage consumption. The process of transformation is based on the empirical wavelet transform (EWT). The encoding process of video data provides secure and less consumption of storage and the reconstruction process consists of the reverse process with the extraction. In this paper, the coding of video is carried out at a very low bit rate with the enhancement of performance by proposing an approach of modified Set Partitioning in Hierarchical Tree (MSPIHT). This method encodes the high frequency frames with the scheduling of wavelet transform for efficient performances of encoding and improves the ability of both the frequency and time. By applying empirical wavelet transform on each video frame, the component of video frequency is extracted and the low frequency frame is encoded by the H.264/AVC standard. The low coefficient values are ignored in applying the threshold and in the reconstruction process, HBLPCE method is used for imaging enhancement. The simulation of the proposed approach analysis shows better performance in reliable process and efficiency when compared to existing.

Introduction to AVS2 Scene Video Coding Techniques

The second generation Audio Video Coding Standard (AVS2) is the most recent video coding standard. By introducing several new coding techniques, AVS2 can provide more efficient compression for scene videos such as surveillance videos, conference videos, etc. Due to the limited scenes, scene videos have great redundancy especially in background region. The new scene video coding techniques applied in AVS2 mainly focus on reducing redundancy in order to achieve higher compression. This paper introduces several important AVS2 scene video coding techniques. Experimental results show that with scene video coding tools, AVS2 can save nearly 40%BD?rate (Bj?ntegaard?Delta bit?rate) on scene videos.

Efficient reconstruction scheme for transform domain Wyner-Ziv video coding

To minimize the errors of the reconstructed values and improve the quality of decoded image, an efficient reconstruction scheme for transform domain Wyner-Ziv （WZ） video coding is proposed. The reconstruction scheme exploits temporal correlation of the coefficient bands, the WZ decoded bits stream and the side information efficiently. When side information is outside the decoded quantization bin, the reconstructed value is derived using expectation of the WZ decoded bit stream and the side information. When side information is within the decoded quantization bin, the reconstructed value is derived using the biased predictor. Simulation results show that the proposed reconstruction scheme gains up to 1.32 dB compared with the commonly used boundary reconstruction scheme at the same bit rates and similar computation cost.

Efficient Multiple Description Scalable Video Coding Scheme Based on Weighted Signal Combinations

A simple, efficient multiple description coding （MDC） algorithm was developed based on weighted signal combinations. The scheme uses the standard video encoder with a pre-processing stage to generate multiple descriptions. The decoder then uses a post-processing algorithm to combine the descriptions to provide better image quality. A scalable codec in the MDC system allows the system to provide multiple descriptions and scalability at the same time. In addition, since the different scalable descriptions may have different qualities, a simple averaging process is not optimal. An optimal weighted combination of the two descriptions was developed based on the signal to noise ratios. Compared with the simple average combination, the algorithm significantly improved the video quality, especially with large quality differences between the two descriptions, with gains of up to 3.56 dB.

A new encoder rate-allocation method for distributed video coding

Distributed video coding(DVC)is a new video coding architecture.Compared with traditional video coding schemes,DVC has a simple encoder and complex decoder,which makes it suitable for circumstances where the encoder equipments are simple but the decoder equipments are complex.Most of the existing DVC architectures use decoder rate allocation method with the help of feedback channel.According to the results of the current decoding round,the decoder informs the encoder by feedback channel whether more parity bits are needed.The use of feedback channel not only increases the system delay,but also limits the use of DVC to instances where there is no feedback channel.In this paper,we propose a novel encoder rate allocation method.First,the simple three-step motion estimation is introduced into the encoder to estimate the side information of the decoder,and then the number of parity bits the decoder needs for each bitplane is allocated at the encoder according to the difference of the estimated side information and the current Wyner-Ziv(WZ)frame.Experiment results indicate that the accuracy of the proposed method is 5.18%-52.93% higher than that of the method proposed by Morbee.

Context-Based 2D-VLC Entropy Coder in AVS Video Coding Standard

In this paper, a Context-based 2D Variable Length Coding （C2DVLC） method for coding the transformed residuals in AVS video coding standard is presented. One feature in C2DVLC is the usage of multiple 2D-VLC tables and another feature is the usage of simple Exponential-Golomb codes. C2DVLC employs context-based adaptive multiple table coding to exploit the statistical correlation between DCT coefficients of one block for higher coding efficiency. ExpGolomb codes are applied to code the pairs of the run-length of zero coefficients and the nonzero coefficient for lower storage requirement. C2DVLC is a low complexity coder in terms of both computational time and memory requirement. The experimental results show that C2DVLC can gain 0.34dB in average for the tested videos when compared with the traditional 2D-VLC coding method like that used in MPEG-2. And compared with CAVLC in H.264/AVC, C2DVLC shows similar coding efficiency.

Performance Comparison of AVS and H.264/AVC Video Coding Standards

A new audio and video compression standard of China,known as advanced Audio Video coding Standard （AVS）.is emerging.This standard provides a technical solution for many applications within the information industry such as digital broadcast,high-density laser—digital storage media,and so on.The basic part of AVS,AVS1-P2,targets standard definition （SD）and high definition（HD）format video compression,and aims to achieve similar coding efficiency as H.264/AVC but with lower computational complexity.In this paper,we first briefly describe the major coding tools in AVS1-P2,and then perform the coding efficiency comparison between AVS1-P2 Jizhun profile and H.264/AVC main profile.The experimental results show that the AVS1-P2 Jizhun profile has an average of 2.96% efficiency loss relative to H.264/AVC main profile in terms of bit-rate saving on HD progressive-scan sequences,and an average of 28.52% coding loss on interlace-scan sequences.Nevertheless,AVS1-P2 possesses a valuable feature of lower computational complexity.

Rate estimate for regular LDPCA codec in distributed video coding

Accumulated low density parity check （LDPCA） codec is proposed for DISCOVER project in distributed video coding （DVC）, which offers flexible coding rate. Although it can use feedback channel to adapt the rate to the correlation of the video, but in real applications, using feedback channel can not always be possible. To solve this problem, some researchers proposed estimating the code rate at the encoder but the performance was not very good. Based on their researches, this paper considers the impact of convergence rate for iteration on rate estimate, which can be calculated using its check matrix. As a pilot study, this paper pays attention to the regular LDPCA codec. At the same time, it considers the impact of deviation in the estimated crossover probability, which gives some constraints to rate estimate. In the experiment, the proposed algorithm can improve the rate-distortion performance by up to 1 dB-1.2 dB.

Robust multiple description distributed video coding using optimized zero-padding

Distributed video coding （DVC） arouses high interests due to its property of low-complexity encoding. This paper proposes a robust multiple description DVC （MDDVC） under the constraint of low-complexity encoding. In MDDVC, zeros are padded to each frame and the resulting big-size video is divided into multiple descriptions. Then, each description is compressed by s hybrid DVC （HDVC） codec and trans- mitted over different channel. When one channel does not work, the lost HDVC description is estimated by the received from other channel, which guarantees the robustness of the system; MDDVC moves the complex motion estimation totally to the decoder so it features low-complexity encoding. In the pre-processing, an optimized zero-padding is also proposed to improve the performance. Experimental results exhibit that the proposed MDDVC scheme achieves better rate-distortion performance and robustness than the referenced especially when packet-loss rate is high.

Chessboard-interpolation-based multiple description video coding

To enhance the robustness of video transmission over noisy channels, this paper presents a multiple description video coding algorithm based on chessboard-interpolation. In the algorithm, the input image is decomposed according to the chessboard pattern, and then interpolated to produce two approximate images with the same resolution. Consequently, the state-of-the-art DCT+MC (Discrete Cosine Transform + Motion Compensation) video codec is independently applied to the two approximate images to generate two descriptions of the original image. In this framework, a fairely good reconstructed image quality is obtained when two descriptions are received simultaneously, while an acceptable reconstructed image quality could be yielded if only one description is available. Moreover, the mismatch between the encoder and the decoder could be effectively controlled through partial coding of the difference signal between two descriptions. In bidirectional video communications, a drift control scheme is further proposed, in which the error drift could be eliminated after the encoder imitating the error concealment actions of the decoder. Since the inherent correlation among adjacent blocks of DCT+MC video coding is efficiently exploited, this algorithm has a better redundancy-rate-distortion (RRD) performance than other multiple description algorithms. Simulation results show that the proposed algorithm is fairly robust while preserves a high compression rate. A more constant reconstructed image quality is achieved over extremely noisy channels, compared with traditional single description coding. In addition, it is observed that the mismatch and the error drift are effectively controlled.