Cloud‐based video streaming services:Trends,challenges,and opportunities

Cloud computing has drastically changed the delivery and consumption of live streaming content.The designs,challenges,and possible uses of cloud computing for live streaming are studied.A comprehensive overview of the technical and business issues surrounding cloudbased live streaming is provided,including the benefits of cloud computing,the various live streaming architectures,and the challenges that live streaming service providers face in delivering high‐quality,real‐time services.The different techniques used to improve the performance of video streaming,such as adaptive bit‐rate streaming,multicast distribution,and edge computing are discussed and the necessity of low‐latency and high‐quality video transmission in cloud‐based live streaming is underlined.Issues such as improving user experience and live streaming service performance using cutting‐edge technology,like artificial intelligence and machine learning are discussed.In addition,the legal and regulatory implications of cloud‐based live streaming,including issues with network neutrality,data privacy,and content moderation are addressed.The future of cloud computing for live streaming is examined in the section that follows,and it looks at the most likely new developments in terms of trends and technology.For technology vendors,live streaming service providers,and regulators,the findings have major policy‐relevant implications.Suggestions on how stakeholders should address these concerns and take advantage of the potential presented by this rapidly evolving sector,as well as insights into the key challenges and opportunities associated with cloud‐based live streaming are provided.

Recent Advances in Video Coding for Machines Standard and Technologies

To improve the performance of video compression for machine vision analysis tasks,a video coding for machines(VCM)standard working group was established to promote standardization procedures.In this paper,recent advances in video coding for machine standards are presented and comprehensive introductions to the use cases,requirements,evaluation frameworks and corresponding metrics of the VCM standard are given.Then the existing methods are presented,introducing the existing proposals by category and the research progress of the latest VCM conference.Finally,we give conclusions.

Early all-zero blocks detecting method for video coding based on novel threshold

In order to decrease both computational complexity and coding time, an improved algorithm for the early detection of all-zero blocks （AZBs） in H. 264/AVC is proposed. The previous AZBs detection algorithms are reviewed. Three types of transformed frequency-domain coefficients, which are quantized to zeros, are analyzed. Based on the three types of frequencydomain scaling factors, the corresponding spatial coefficients are derived. Then the Schwarz inequality is applied to the derivation of the three thresholds based on spatial coefficients. Another threshold is set on the basis of the probability distribution of zero coefficients in a block. As a result, an adaptive AZBs detection algorithm is proposed based on the minimum of the former three thresholds and the threshold of zero blocks distribution. The simulation results show that, compared with the existing AZBs detection algorithms, the proposed algorithm achieves a 5% higher detection ratio in AZBs and 4% to 10% computation saving with only 0. 1 dB video quality degradation.

基于块编码特点的压缩视频质量增强算法

针对现有压缩视频质量增强算法未能充分利用压缩视频特点的问题,研究了视频编码与压缩视频质量增强任务之间的本质关系,并针对性地设计了一种基于三维卷积神经网络(3D convolutional neural network, 3D-CNN)的非对齐压缩视频质量增强算法。实验结果表明:相较于高效视频编码(high efficiency video coding, HEVC)标准H.265,所提算法在低延迟(low delay, LD)配置下且量化参数(quantization parameter, QP)为37时,峰值信噪比(peak signal-to-noise ratio, PSNR)提升了0.465 2 dB;相较于数据压缩会议(data compression conference, DCC)中提出的多帧引导的注意力网络(multi-frame guided attention network, MGANet)方法,该算法PSNR的增长量提升了15.1%。

Perceptual video coding method based on JND and AR model

In order to achieve better perceptual coding quality while using fewer bits, a novel perceptual video coding method based on the just-noticeable-distortion （JND） model and the auto-regressive （AR） model is explored. First, a new texture segmentation method exploiting the JND profile is devised to detect and classify texture regions in video scenes. In this step, a spatial-temporal JND model is proposed and the JND energy of every micro-block unit is computed and compared with the threshold. Secondly, in order to effectively remove temporal redundancies while preserving high visual quality, an AR model is applied to synthesize the texture regions. All the parameters of the AR model are obtained by the least-squares method and each pixel in the texture region is generated as a linear combination of pixels taken from the closest forward and backward reference frames. Finally, the proposed method is compared with the H.264/AVC video coding system to demonstrate the performance. Various sequences with different types of texture regions are used in the experiment and the results show that the proposed method can reduce the bit-rate by 15% to 58% while maintaining good perceptual quality.

A rate control scheme for H.264 video under low bandwidth channel

The dilemma of the quantization parameter (QP) being involved in both rate control and rate-distortion optimization (RDO) prevents using the traditional rate control scheme. Although some rate control schemes are proposed to circumvent the dilemma, the inaccurate prediction model and improper bit allocation deter H.264 application on low bandwidth channel. To resolve this issue, this paper proposes a novel rate control scheme by considering the macroblock (MB) encoding complexity variation and buffer variation and by exploiting the spatio-temporal correlation sufficiently well. Simulations showed that this scheme improves the perceptual quality of the pictures with similar or smaller PSNR deviations when compared to that of rate control in JVT-O016.

Low Bit Rate Underwater Video Image Compression and Coding Method Based on Wavelet Decomposition

In view of the limited bandwidth of underwater video image transmission,a low bit rate underwater video compression coding method is proposed.Based on the preprocessing process of wavelet transform and coefficient down-sampling,the visual redundancy of underwater image is removed and the computational coefficients and coding bits are reduced.At the same time,combined with multi-level wavelet decomposition,inter frame motion compensation,entropy coding and other methods,according to the characteristics of different types of frame image data,reduce the number of calculations and improve the coding efficiency.The experimental results show that the reconstructed image quality can meet the visual requirements,and the average compression ratio of underwater video can meet the requirements of underwater acoustic channel transmission rate.

Overview of the Second Generation AVS Video Coding Standard(AVS2)

AVS2 is a new generation video coding standard developed by the AVS working group. Compared with the first generation AVS video coding standard, known as AVS1, AVS2 significantly improves coding performance by using many new coding technologies, e.g., adaptive block partition and two level transform coding. Moreover, for scene video, e.g. surveillance video and conference vid?eo, AVS2 provided a background picture modeling scheme to achieve more accurate prediction, which can also make object detec?tion and tracking in surveillance video coding more flexible. Experimental results show that AVS2 is competitive with High Effi?ciency Video Coding (HEVC) in terms of performance. Especially for scene video, AVS2 can achieve 39% bit rate saving over HEVC.

RATE CONTROL ALGORITHM FOR H.264 VIDEO ENCODER

This letter proposes a rate control algorithm for H.264 video encoder, which is based on block activity and buffer state. Experimental results indicate that it has an excellent performance by providing much accurate bit rate and better coding efficiency compared with H.264. The computational complexity of the algorithm is reduced by adopting a novel block activity description method using the Sum of Absolute Difference (SAD) of 16× 16 mode, and its robustness is enhanced by introducing a feedback circuit at frame layer.

Color compensation for multi-view video coding based on diversity of cameras

A novel color compensation method for multi-view video coding (MVC) is proposed, which efficiently exploits the inter-view dependencies between views with the existence of color mismatch caused by the diversity of cameras. A color compensation model is developed in RGB channels and then extended to YCbCr channels for practical use. A modified inter-view reference picture is constructed based on the color compensation model, which is more similar to the coding picture than the original inter-view reference picture. Moreover, the color compensation factors can be derived in both encoder and decoder, therefore no additional data need to be transmitted to the decoder. The experimental results show that the proposed method improves the coding efficiency of MVC and maintains good subjective quality.

Rate Control Algorithm of Wireless Video Based on Visual Saliency Map Model

In order to further improve the efficiency of video compression, we introduce a perceptual characteristics of Human Visual System (HVS) to video coding, and propose a novel video coding rate control algorithm based on human visual saliency model in H.264/AVC. Firstly, we modifie Itti's saliency model. Secondly, target bits of each frame are allocated through the correlation of saliency region between the current and previous frame, and the complexity of each MB is modified through the saliency value and its Mean Absolute Difference (MAD) value. Lastly, the algorithm was implemented in JVT JM12.2. Simulation results show that, comparing with traditional rate control algorithm, the proposed one can reduce the coding bit rate and improve the reconstructed video subjective quality, especially for visual saliency region. It is very suitable for wireless video transmission.

Cooperative Caching for Scalable Video Coding Using Value-Decomposed Dimensional Networks

Scalable video coding(SVC)has been widely used in video-on-demand(VOD)service,to efficiently satisfy users’different video quality requirements and dynamically adjust video stream to timevariant wireless channels.Under the 5G network structure,we consider a cooperative caching scheme inside each cluster with SVC to economically utilize the limited caching storage.A novel multi-agent deep reinforcement learning(MADRL)framework is proposed to jointly optimize the video access delay and users’satisfaction,where an aggregation node is introduced helping individual agents to achieve global observations and overall system rewards.Moreover,to cope with the large action space caused by the large number of videos and users,a dimension decomposition method is embedded into the neural network in each agent,which greatly reduce the computational complexity and memory cost of the reinforcement learning.Experimental results show that:1)the proposed value-decomposed dimensional network(VDDN)algorithm achieves an obvious performance gain versus the traditional MADRL;2)the proposed VDDN algorithm can handle an extremely large action space and quickly converge with a low computational complexity.

Frame-layer bit allocation for multi-view video coding based on frame complexity estimation

Current multi-view video coding （MVC） reference model in joint video team （JVT） does not provide efficient rate control schemes. This paper presents a rate control algorithm for MVC by improving the quadratic rate-distortion （R-D） model. We reasonably allocate bit-rate among views based on the correlation analysisl The proposed algorithm consists of three levels to control the rate bits more accurately, of which the frame layer allocates bits according to the frame complexity and the temporal activity. Extensive experiments show that the proposed algorithm can control the bit rate efficiently.

AN EFFICIENT 3-DIMENSIONAL DISCRETE WAVELET TRANSFORM ARCHITECTURE FOR VIDEO PROCESSING APPLICATION

This paper presents an optimized 3-D Discrete Wavelet Transform (3-DDWT) architecture. 1-DDWT employed for the design of 3-DDWT architecture uses reduced lifting scheme approach. Further the architecture is optimized by applying block enabling technique, scaling, and rounding of the filter coefficients. The proposed architecture uses biorthogonal (9/7) wavelet filter. The architecture is modeled using Verilog HDL, simulated using ModelSim, synthesized using Xilinx ISE and finally implemented on Virtex-5 FPGA. The proposed 3-DDWT architecture has slice register utilization of 5%, operating frequency of 396 MHz and a power consumption of 0.45 W.

In-Network Puncturing for Delay-Efficient Rate Control of Distributed Video Coding in Wireless Video Sensor Networks

Wyner-Ziv Video Coding （WZVC） is considered as a promising video coding scheme for Wireless Video Sensor Networks （WVSNs） due to its high compression efficiency and error resilience functionalities, as well as its low encoding complex- ity. To achieve a good Rate-Distortion （R-D） per- formance, the current WZVC paradi^prls usually a- dopt an end-to-end rate control scheme in which the decoder repeatedly requests the additional deco- ding data from the encoder for decoding Wyner-Ziv frames. Therefore, the waiting time of the additional decoding data is especially long in multihop WVSNs. In this paper, we propose a novel pro- gressive in-network rate control scheme for WZVC. The proposed in-network puncturing-based rate control scheme transfers the partial channel codes puncturing task from the encoder to the relay nodes. Then, the decoder can request the addition- al decoding data from the relay nodes instead of the encoder, and the total waiting time for deco- ding Wyner-Ziv frames is reduced consequently. Simulation results validate the proposed rate con- trol scheme.

A WYNER-ZIV VIDEO CODING METHOD UTILIZING MIXTURE CORRELATION NOISE MODEL

In Wyner-Ziv (WZ) Distributed Video Coding (DVC), correlation noise model is often used to describe the error distribution between WZ frame and the side information. The accuracy of the model can influence the performance of the video coder directly. A mixture correlation noise model in Discrete Cosine Transform (DCT) domain for WZ video coding is established in this paper. Different correlation noise estimation method is used for direct current and alternating current coefficients. Parameter estimation method based on expectation maximization algorithm is used to estimate the Laplace distribution center of direct current frequency band and Mixture Laplace-Uniform Distribution Model (MLUDM) is established for alternating current coefficients. Experimental results suggest that the proposed mixture correlation noise model can describe the heavy tail and sudden change of the noise accurately at high rate and make significant improvement on the coding efficiency compared with the noise model presented by DIStributed COding for Video sERvices (DISCOVER).

Error concealment in pixel based Wyner-Ziv video coding

In the Wyner-Ziv(WZ) video coding paradigm, a virtual correlation channel is assumed between the quantized source and the side information(SI) at the decoder, and channel coding is applied to achieve compression. In this paper, errors caused by the virtual correlation channel are addressed and an error concealment approach is proposed for pixel-based WZ video coding. In the approach, errors after decoding are classified into two types. Type 1 errors are caused by residual bit errors after channel decoding, while type 2 errors are due to low quality of SI in part of a frame which causes SI not lying within the quantization bin of a decoded quantized pixel value. Two separate strategies are respectively designed to detect and conceal the two types of errors. Simulations are carried out and results are presented to demonstrate the effectiveness of the proposed approach.

Introduction to AVS2 Scene Video Coding Techniques

The second generation Audio Video Coding Standard (AVS2) is the most recent video coding standard. By introducing several new coding techniques, AVS2 can provide more efficient compression for scene videos such as surveillance videos, conference videos, etc. Due to the limited scenes, scene videos have great redundancy especially in background region. The new scene video coding techniques applied in AVS2 mainly focus on reducing redundancy in order to achieve higher compression. This paper introduces several important AVS2 scene video coding techniques. Experimental results show that with scene video coding tools, AVS2 can save nearly 40%BD?rate (Bj?ntegaard?Delta bit?rate) on scene videos.

Improving Low Bitrate Video Coding via Computation Incorporating A Priori Information

The growing number of mobile users, as well as the diversification in types of services have resulted in increasing demands for wireless network bandwidth in recent years. Although evolving transmission techniques are able to enlarge the network capacity to some degree, they still cannot satisfy the requirements of mobile users. Meanwhile, following Moore's Law, the data processing capabilities of mobile user terminals are continuously improving. In this paper, we explore possible methods of trading strong computational power at wireless terminals for transmission efficiency of communications. Taking the specific scenario of wireless video conversation, we propose a model-based video coding scheme by learning the structures in multimedia contents. Benefiting from both strong computing capability and pre-learned model priors, only low-dimensional parameters need to be transmitted; and the intact multimedia contents can also be reconstructed at the receivers in real-time. Experiment results indicate that, compared to conventional video codecs, the proposed scheme significantly reduces the data rate with the aid of computational capability at wireless terminals.

AVS 3D Video Coding Technology and System

Following the success of the audio video standard （AVS） for 2D video coding, in 2008, the China AVS workgroup started developing 3D video （3DV） coding techniques. In this paper, we discuss the background, technical features, and applications of AVS 3DV coding technology. We introduce two core techniques used in AVS 3DV coding： inter-view prediction and enhanced stereo packing coding. We elaborate on these techniques, which are used in the AVS real-time 3DV encoder. An application of the AVS 3DV coding system is presented to show the great practical value of this system. Simulation results show that the advanced techniques used in AVS 3DV coding provide remarkable coding gain compared with techniques used in a simulcast scheme.