Hierarchical Search Motion Estimation Algorithms for Real-time Video Coding

ＨｉｅｒａｒｃｈｉｃａｌＳｅａｒｃｈＭｏｔｉｏｎＥｓｔｉｍａｔｉｏｎＡｌｇｏｒｉｔｈｍｓｆｏｒＲｅａｌｔｉｍｅＶｉｄｅｏＣｏｄｉｎｇＺｈａｎｇＹｏｎｇＹｕＬｕＺｈｏｕＦｅｎｇＹａｏＱｉｎｇｄｏｎｇ（ＺｈｅｊｉａｎｇＵｎｉｖｅｒｓｉｔｙ，Ｈａｎｇｚｈ...

Two-Phase Rate Adaptation Strategy for Improving Real-Time Video QoE in Mobile Networks

With the popularity of smart handheld devices, mobile streaming video has multiplied the global network traffic in recent years. A huge concern of users' quality of experience(Qo E) has made rate adaptation methods very attractive. In this paper, we propose a two-phase rate adaptation strategy to improve users' real-time video Qo E. First, to measure and assess video Qo E, we provide a continuous Qo E prediction engine modeled by RNN recurrent neural network. Different from traditional Qo E models which consider the Qo E-aware factors separately or incompletely, our RNN-Qo E model accounts for three descriptive factors(video quality, rebuffering, and rate change) and reflects the impact of cognitive memory and recency. Besides, the video playing is separated into the initial startup phase and the steady playback phase, and we takes different optimization goals for each phase: the former aims at shortening the startup delay while the latter ameliorates the video quality and the rebufferings. Simulation results have shown that RNN-Qo E can follow the subjective Qo E quite well, and the proposed strategy can effectively reduce the occurrence of rebufferings caused by the mismatch between the requested video rates and the fluctuated throughput and attains standout performance on real-time Qo E compared with classical rate adaption methods.

Real-time Audio &Video Transmission System Based on Visible Light Communication

With the increasing popularity of solid sate lighting devices, Visible Light Communication (VLC) is globally recognized as an advanced and promising technology to realize short-range, high speed as well as large capacity wireless data transmission. In this paper, we propose a prototype of real-time audio and video broadcast system using inexpensive commercially available light emitting diode (LED) lamps. Experimental results show that real-time high quality audio and video with the maximum distance of 3 m can be achieved through proper layout of LED sources and improvement of concentration effects. Lighting model within room environment is designed and simulated which indicates close relationship between layout of light sources and distribution of illuminance.

Recent Advances in Video Coding for Machines Standard and Technologies

To improve the performance of video compression for machine vision analysis tasks,a video coding for machines(VCM)standard working group was established to promote standardization procedures.In this paper,recent advances in video coding for machine standards are presented and comprehensive introductions to the use cases,requirements,evaluation frameworks and corresponding metrics of the VCM standard are given.Then the existing methods are presented,introducing the existing proposals by category and the research progress of the latest VCM conference.Finally,we give conclusions.

Adaptive Hybrid Forward Error Correction Coding Scheme for Video Transmission

This paper proposes an adaptive hybrid forward error correction(AH-FEC)coding scheme for coping with dynamic packet loss events in video and audio transmission.Specifically,the proposed scheme consists of a hybrid Reed-Solomon and low-density parity-check(RS-LDPC)coding system,combined with a Kalman filter-based adaptive algorithm.The hybrid RS-LDPC coding accommodates a wide range of code length requirements,employing RS coding for short codes and LDPC coding for medium-long codes.We delimit the short and medium-length codes by coding performance so that both codes remain in the optimal region.Additionally,a Kalman filter-based adaptive algorithm has been developed to handle dynamic alterations in a packet loss rate.The Kalman filter estimates packet loss rate utilizing observation data and system models,and then we establish the redundancy decision module through receiver feedback.As a result,the lost packets can be perfectly recovered by the receiver based on the redundant packets.Experimental results show that the proposed method enhances the decoding performance significantly under the same redundancy and channel packet loss.

Real-time programmable coding metasurface antenna for multibeam switching and scanning

Novel electromagnetic wave modulation by programmable dynamic metasurface promotes the device design freedom,while multibeam antennas have sparked tremendous interest in wireless communications.A programmable coding antenna based on active metasurface elements(AMSEs)is proposed in this study,allowing scanning and state switching of multiple beams in real time.To obtain the planar array phase distribution in quick response,the aperture field superposition and discretization procedures are investigated.Without the need for a massive algorithm or elaborate design,this electronically controlled antenna with integrated radiation and phase-shift functions can flexibly manipulate the scattering state of multiple beams under field-programmable gate array(FPGA)control.Simulation and experimental results show that the multiple directional beams dynamically generated in the metasurface upper half space have good radiation performance,with the main lobe directions closely matching the predesigned angles.This metasurface antenna has great potential for future applications in multitarget radar,satellite navigation,and reconfigurable intelligent metasurfaces.

Priority-Based Real-Time Stream Coding over Multi-Channel with Packet Erasures

Multi-channel can be used to provide higher transmission ability to the bandwidth-intensive and delay-sensitive real-time streams. However, traditional channel capacity theories and coding schemes are seldom designed for the real-time streams with strict delay constraint, especially in multi-channel context. This paper considers a real-time stream system, where real-time messages with different importance should be transmitted through several packet erasure channels, and be decoded by the receiver within a fixed delay. Based on window erasure channels and i.i.d.(identically and independently distributed) erasure channels, we derive the Multi-channel Real-time Stream Transmission(MRST) capacity models for Symmetric Real-time(SR) streams and Asymmetric Real-time(AR) streams respectively. Moreover, for window erasures, a Maximum Equilibrium Intra-session Code(MEIC) is presented for SR and AR streams, and is shown able to asymptotically achieve the theoretical MRST capacity. For i.i.d. erasures, we propose an Adaptive Maximum Equilibrium Intra-session Code(AMEIC), and then prove AMEIC can closely approach the MRST transmission capacity. Finally, the performances of the proposed codes are verified by simulations.

APPLICATION OF TURBO CODES IN HIGH-SPEED REAL-TIME CHANNEL

The time delay of Turbo codes due to its iterative decoding is the main bottleneck of its application in real-time channel. However, the time delay can be greatly shortened through the adoption of parallel decod-ing algorithm, dividing the received bits into several sub-blocks and processing in parallel. This letter mainly discusses the applicability of turbo codes in high-speed real-time channel through the study of a parallel turbo decoding algorithm based on 3GPP-proposed turbo encoder and interleaver in various channel. Simulation re-sult shows that, by choosing an appropriate sub-block length, the time delay can be obviously shortened with-out degrading the performance and increasing hardware complexity, and furthermore indicates the applicability of Turbo codes in high-speed real-time channel.

Distributed Video Coding Using LDPC Codes for Wireless Video

Popular video coding standards like H.264 and MPEG working on the principle of motion-compensated pre-dictive coding demand much of the computational resources at the encoder increasing its complexity. Such bulky encoders are not suitable for applications like wireless low power surveillance, multimedia sensor networks, wireless PC cameras, mobile camera phones etc. New video coding scheme based on the principle of distributed source coding is looked upon in this paper. This scheme supports a low complexity encoder, at the same time trying to achieve the rate distortion performance of conventional video codecs. Current im-plementation uses LDPC codes for syndrome coding.

The Confidentiality of Coding Video Games with Cheat Code and Bots for Cheating in a Virtual World

Video games have been around for several decades and have had many advancements from the original start of video games. Video games started as virtual games that were advertised towards children, and these virtual games created a virtual reality of a variety of genres. These genres included sports games, such as tennis, football, baseball, war games, fantasy, puzzles, etc. The start of these games was derived from a sports genre and now has a popularity in multiplayer-online-shooting games. The purpose of this paper is to investigate different types of tools available for cheating in virtual world making players have undue advantage over other players in a competition. With the advancement in technology, these video games have become more expanded in the development aspects of gaming. Video game developers have created long lines of codes to create a new look of video games. As video games have progressed, the coding, bugs, bots, and errors of video games have changed throughout the years. The coding of video games has branched out from the original video games, which have given many benefits to this virtual world, while simultaneously creating more problems such as bots. Analysis of tools available for cheating in a game has disadvantaged normal gamer in a fair contest.

Early all-zero blocks detecting method for video coding based on novel threshold

In order to decrease both computational complexity and coding time, an improved algorithm for the early detection of all-zero blocks （AZBs） in H. 264/AVC is proposed. The previous AZBs detection algorithms are reviewed. Three types of transformed frequency-domain coefficients, which are quantized to zeros, are analyzed. Based on the three types of frequencydomain scaling factors, the corresponding spatial coefficients are derived. Then the Schwarz inequality is applied to the derivation of the three thresholds based on spatial coefficients. Another threshold is set on the basis of the probability distribution of zero coefficients in a block. As a result, an adaptive AZBs detection algorithm is proposed based on the minimum of the former three thresholds and the threshold of zero blocks distribution. The simulation results show that, compared with the existing AZBs detection algorithms, the proposed algorithm achieves a 5% higher detection ratio in AZBs and 4% to 10% computation saving with only 0. 1 dB video quality degradation.

Perceptual video coding method based on JND and AR model

In order to achieve better perceptual coding quality while using fewer bits, a novel perceptual video coding method based on the just-noticeable-distortion （JND） model and the auto-regressive （AR） model is explored. First, a new texture segmentation method exploiting the JND profile is devised to detect and classify texture regions in video scenes. In this step, a spatial-temporal JND model is proposed and the JND energy of every micro-block unit is computed and compared with the threshold. Secondly, in order to effectively remove temporal redundancies while preserving high visual quality, an AR model is applied to synthesize the texture regions. All the parameters of the AR model are obtained by the least-squares method and each pixel in the texture region is generated as a linear combination of pixels taken from the closest forward and backward reference frames. Finally, the proposed method is compared with the H.264/AVC video coding system to demonstrate the performance. Various sequences with different types of texture regions are used in the experiment and the results show that the proposed method can reduce the bit-rate by 15% to 58% while maintaining good perceptual quality.

RTP payload format for H.264/SVC scalable video coding

The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team’s work. In its present working draft, the higher level syntax of SVC follows the design principles of H.264/AVC. Self-contained network abstraction layer units (NAL units) form natural entities for packetization. The SVC specification is by no means finalized yet, but nevertheless the work towards an optimized RTP payload format has already started. RFC 3984, the RTP payload specification for H.264/AVC has been taken as a starting point, but it became quickly clear that the scalable features of SVC require adaptation in at least the areas of capability/operation point signaling and documentation of the extended NAL unit header. This paper first gives an overview of the history of scalable video coding, and then reviews the video coding layer (VCL) and NAL of the latest SVC draft specification. Finally, it discusses different aspects of the draft SVC RTP payload format, in- cluding the design criteria, use cases, signaling and payload structure.

Deadline-aware network coding for video on demand service over P2P networks

We are interested in providing Video-on-Demand (VoD) streaming service to a large population of clients using peer-to-peer (P2P) approach. Given the asynchronous demands from multiple clients, continuously changing of the buffered contents, and the continuous video display requirement, how to collaborate with potential partners to get expected data for future content delivery are very important and challenging. In this paper, we develop a novel scheduling algorithm based on deadline- aware network coding (DNC) to fully exploit the network resource for efficient VoD service. DNC generalizes the existing net- work coding (NC) paradigm, an elegant solution for ubiquitous data distribution. Yet, with deadline awareness, DNC improves the network throughput and meanwhile avoid missing the play deadline in high probability, which is a major deficiency of the con- ventional NC. Extensive simulation results demonstrated that DNC achieves high streaming continuity even in tight network conditions.

Overview of the Second Generation AVS Video Coding Standard(AVS2)

AVS2 is a new generation video coding standard developed by the AVS working group. Compared with the first generation AVS video coding standard, known as AVS1, AVS2 significantly improves coding performance by using many new coding technologies, e.g., adaptive block partition and two level transform coding. Moreover, for scene video, e.g. surveillance video and conference vid?eo, AVS2 provided a background picture modeling scheme to achieve more accurate prediction, which can also make object detec?tion and tracking in surveillance video coding more flexible. Experimental results show that AVS2 is competitive with High Effi?ciency Video Coding (HEVC) in terms of performance. Especially for scene video, AVS2 can achieve 39% bit rate saving over HEVC.

A Novel Objective Quality Assessment Method for Video Conferencing Coding

Most recently, due to the demand of immersive communication, region-of-interest-based(ROI) high efficiency video coding(HEVC) approaches in conferencing scenarios have become increasingly important. However, there exists no objective metric, specially developed for efficiently evaluating the perceived visual quality of video conferencing coding. Therefore, this paper proposes a novel objective quality assessment method, namely Gaussian mixture model based peak signal-tonoise ratio(GMM-PSNR), for the perceptual video conferencing coding. First, eye tracking experiments, together with a real-time technique of face and facial feature extraction, are introduced. In the experiments, importance of background, face, and facial feature regions is identified, and it is then quantified based on eye fixation points over test videos. Next, assuming that the distribution of the eye fixation points obeys Gaussian mixture model, we utilize expectation-maximization(EM) algorithm to generate an importance weight map for each frame of video conferencing coding, in light of a new term eye fixation points/pixel(efp/p). According to the generated weight map, GMM-PSNR is developed for quality assessment by assigning different weights to the distortion of each pixel in the video frame. Finally, we utilize some experiments to investigate the correlation of the proposed GMM-PSNR and other conventional objective metrics with subjective quality metrics. The experimental results show the effectiveness of GMM-PSNR.

Video over IP using standard-compatible multiple description coding: an IETF proposal

Standard-compatible multiple description coding (MDC) and layered coding (LC) are efficient ways to ensure erasure resilient, scalable transmission of encoded multimedia sources via RTP, allowing a gradual degradation of the application quality with increasing packet loss rate and decreasing bandwidth/throughput on the network. In this paper we review the stan- dard-compatible framework proposed to IETF. Alternative techniques such as robust source coding and channel coding techniques (ARQ: automatic repeat request, FEC: forward error correction) are presented; their integration into the proposed framework is also discussed. The performances of MDC and LC either coupled with channel coding or not, are summarized by reference to current literature. Typical cases and examples are illustrated.

Multipoint videoconferencing with scalable video coding

We describe a system for multipoint videoconferencing that offers extremely low end-to-end delay, low cost and complexity, and high scalability, alongside standard features associated with high-end solutions such as rate matching and per- sonal video layout. The system accommodates heterogeneous receivers and networks based on the Internet Protocol and relies on scalable video coding to provide a coded representation of a source video signal at multiple temporal and spatial resolutions as well as quality levels. These are represented by distinct bitstream components which are created at each end-user encoder. Depending on the specific conferencing environment, some or all of these components are transmitted to a Scalable Video Conferencing Server (SVCS). The SVCS redirects these components to one or more recipients depending on, e.g., the available network con- ditions and user preferences. The scalable aspect of the video coding technique allows the system to adapt to different network conditions, and also accommodates different end-user requirements (e.g., a user may elect to view another user at a high or low spatial resolution). Performance results concerning flexibility, video quality and delay of the system are presented using the Joint Scalable Video Model (JSVM) of the forthcoming SVC (H.264 Annex G) standard, demonstrating that scalable coding outper- forms existing state-of-the-art systems and offers the right platform for building next-generation multipoint videoconferencing systems.

Low Bit Rate Underwater Video Image Compression and Coding Method Based on Wavelet Decomposition

In view of the limited bandwidth of underwater video image transmission,a low bit rate underwater video compression coding method is proposed.Based on the preprocessing process of wavelet transform and coefficient down-sampling,the visual redundancy of underwater image is removed and the computational coefficients and coding bits are reduced.At the same time,combined with multi-level wavelet decomposition,inter frame motion compensation,entropy coding and other methods,according to the characteristics of different types of frame image data,reduce the number of calculations and improve the coding efficiency.The experimental results show that the reconstructed image quality can meet the visual requirements,and the average compression ratio of underwater video can meet the requirements of underwater acoustic channel transmission rate.

Cooperative Caching for Scalable Video Coding Using Value-Decomposed Dimensional Networks

Scalable video coding(SVC)has been widely used in video-on-demand(VOD)service,to efficiently satisfy users’different video quality requirements and dynamically adjust video stream to timevariant wireless channels.Under the 5G network structure,we consider a cooperative caching scheme inside each cluster with SVC to economically utilize the limited caching storage.A novel multi-agent deep reinforcement learning(MADRL)framework is proposed to jointly optimize the video access delay and users’satisfaction,where an aggregation node is introduced helping individual agents to achieve global observations and overall system rewards.Moreover,to cope with the large action space caused by the large number of videos and users,a dimension decomposition method is embedded into the neural network in each agent,which greatly reduce the computational complexity and memory cost of the reinforcement learning.Experimental results show that:1)the proposed value-decomposed dimensional network(VDDN)algorithm achieves an obvious performance gain versus the traditional MADRL;2)the proposed VDDN algorithm can handle an extremely large action space and quickly converge with a low computational complexity.