Estimating Reduced-Reference Video Quality for Quality-Based Streaming Video

Reduced-reference （RR） video-quality estimators send a small signature to the receiver. This signature comprises the original video content as well as the video stream. RR quality estimation provides reliability and involves a small data payload. While significant in theory, RR estimators have only recently been used in practice for quality monitoring and adaptive system con- trol in streaming-video frameworks. In this paper, we classify RR algorithms according to whether they are based on a） model- ing the signal distortion, b） modeling the human visual system, or c） analyzing the video signal source. We review proposed RR techniques for monitoring and controlling quality in streaming video systems.

Machine Learning Based Classifiers for QoE Prediction Framework in Video Streaming over 5G Wireless Networks

Recently,the combination of video services and 5G networks have been gaining attention in the wireless communication realm.With the brisk advancement in 5G network usage and the massive popularity of threedimensional video streaming,the quality of experience(QoE)of video in 5G systems has been receiving overwhelming significance from both customers and service provider ends.Therefore,effectively categorizing QoE-aware video streaming is imperative for achieving greater client satisfaction.This work makes the following contribution:First,a simulation platform based on NS-3 is introduced to analyze and improve the performance of video services.The simulation is formulated to offer real-time measurements,saving the expensive expenses associated with real-world equipment.Second,A valuable framework for QoE-aware video streaming categorization is introduced in 5G networks based on machine learning(ML)by incorporating the hyperparameter tuning(HPT)principle.It implements an enhanced hyperparameter tuning(EHPT)ensemble and decision tree(DT)classifier for video streaming categorization.The performance of the ML approach is assessed by considering precision,accuracy,recall,and computation time metrics for manifesting the superiority of these classifiers regarding video streaming categorization.This paper demonstrates that our ML classifiers achieve QoE prediction accuracy of 92.59%for(EHPT)ensemble and 87.037%for decision tree(DT)classifiers.

Adaptive Learning Video Streaming with QoE in Multi-Home Heterogeneous Networks

In recent years,real-time video streaming has grown in popularity.The growing popularity of the Internet of Things(IoT)and other wireless heterogeneous networks mandates that network resources be carefully apportioned among versatile users in order to achieve the best Quality of Experience(QoE)and performance objectives.Most researchers focused on Forward Error Correction(FEC)techniques when attempting to strike a balance between QoE and performance.However,as network capacity increases,the performance degrades,impacting the live visual experience.Recently,Deep Learning(DL)algorithms have been successfully integrated with FEC to stream videos across multiple heterogeneous networks.But these algorithms need to be changed to make the experience better without sacrificing packet loss and delay time.To address the previous challenge,this paper proposes a novel intelligent algorithm that streams video in multi-home heterogeneous networks based on network-centric characteristics.The proposed framework contains modules such as Intelligent Content Extraction Module(ICEM),Channel Status Monitor(CSM),and Adaptive FEC(AFEC).This framework adopts the Cognitive Learning-based Scheduling(CLS)Module,which works on the deep Reinforced Gated Recurrent Networks(RGRN)principle and embeds them along with the FEC to achieve better performances.The complete framework was developed using the Objective Modular Network Testbed in C++(OMNET++),Internet networking(INET),and Python 3.10,with Keras as the front end and Tensorflow 2.10 as the back end.With extensive experimentation,the proposed model outperforms the other existing intelligentmodels in terms of improving the QoE,minimizing the End-to-End Delay(EED),and maintaining the highest accuracy(98%)and a lower Root Mean Square Error(RMSE)value of 0.001.

Workout Action Recognition in Video Streams Using an Attention Driven Residual DC-GRU Network

Regular exercise is a crucial aspect of daily life, as it enables individuals to stay physically active, lowers thelikelihood of developing illnesses, and enhances life expectancy. The recognition of workout actions in videostreams holds significant importance in computer vision research, as it aims to enhance exercise adherence, enableinstant recognition, advance fitness tracking technologies, and optimize fitness routines. However, existing actiondatasets often lack diversity and specificity for workout actions, hindering the development of accurate recognitionmodels. To address this gap, the Workout Action Video dataset (WAVd) has been introduced as a significantcontribution. WAVd comprises a diverse collection of labeled workout action videos, meticulously curated toencompass various exercises performed by numerous individuals in different settings. This research proposes aninnovative framework based on the Attention driven Residual Deep Convolutional-Gated Recurrent Unit (ResDCGRU)network for workout action recognition in video streams. Unlike image-based action recognition, videoscontain spatio-temporal information, making the task more complex and challenging. While substantial progresshas been made in this area, challenges persist in detecting subtle and complex actions, handling occlusions,and managing the computational demands of deep learning approaches. The proposed ResDC-GRU Attentionmodel demonstrated exceptional classification performance with 95.81% accuracy in classifying workout actionvideos and also outperformed various state-of-the-art models. The method also yielded 81.6%, 97.2%, 95.6%, and93.2% accuracy on established benchmark datasets, namely HMDB51, Youtube Actions, UCF50, and UCF101,respectively, showcasing its superiority and robustness in action recognition. The findings suggest practicalimplications in real-world scenarios where precise video action recognition is paramount, addressing the persistingchallenges in the field. TheWAVd dataset serves as a catalyst for the development ofmore robust and effective fitnesstracking systems and ultimately promotes healthier lifestyles through improved exercise monitoring and analysis.

MEC-Assisted Flexible Transcoding Strategy for Adaptive Bitrate Video Streaming in Small Cell Networks

Adaptive bitrate video streaming(ABR)has become a critical technique for mobile video streaming to cope with time-varying network conditions and different user preferences.However,there are still many problems in achieving high-quality ABR video streaming over cellular networks.Mobile Edge Computing(MEC)is a promising paradigm to overcome the above problems by providing video transcoding capability and caching the ABR video streaming within the radio access network(RAN).In this paper,we propose a flexible transcoding strategy to provide viewers with low-latency video streaming services in the MEC networks under the limited storage,computing,and spectrum resources.According to the information collected from users,the MEC server acts as a controlling component to adjust the transcoding strategy flexibly based on optimizing the video caching placement strategy.Specifically,we cache the proper bitrate version of the video segments at the edge servers and select the appropriate bitrate version of the video segments to perform transcoding under jointly considering access control,resource allocation,and user preferences.We formulate this problem as a nonconvex optimization and mixed combinatorial problem.Moreover,the simulation results indicate that our proposed algorithm can ensure a low-latency viewing experience for users.

Delay-Based Cross-Layer QoS Scheme for Video Streaming in Wireless Ad Hoc Networks

With correlating with human perception, quality of experience(Qo E) is also an important measurement in evaluation of video quality in addition to quality of service(Qo S). A cross-layer scheme based on Lyapunov optimization framework for H.264/AVC video streaming over wireless Ad hoc networks is proposed, with increasing both Qo E and Qo S performances. Different from existing works, this scheme routes and schedules video packets according to the statuses of the frame buffers at the destination nodes to reduce buffer underflows and to increase video playout continuity. The waiting time of head-ofline packets of data queues are considered in routing and scheduling to reduce the average end-to-end delay of video sessions. Different types of packets are allocated with different priorities according to their generated rates under H.264/AVC. To reduce the computational complexity, a distributed media access control policy and a power control algorithm cooperating with the media access policy are proposed. Simulation results show that, compared with existing schemes, this scheme can improve both the Qo S and Qo E performances. The average peak signal-to-noise ratio(PSNR) of the received video streams is also increased.

Efficient Computation Offloading in Mobile Cloud Computing for Video Streaming Over 5G

In this paper,we investigate video quality enhancement using computation offloading to the mobile cloud computing(MCC)environment.Our objective is to reduce the computational complexity required to covert a low-resolution video to high-resolution video while minimizing computation at the mobile client and additional communication costs.To do so,we propose an energy-efficient computation offloading framework for video streaming services in a MCC over the fifth generation(5G)cellular networks.In the proposed framework,the mobile client offloads the computational burden for the video enhancement to the cloud,which renders the side information needed to enhance video without requiring much computation by the client.The cloud detects edges from the upsampled ultra-high-resolution video(UHD)and then compresses and transmits them as side information with the original low-resolution video(e.g.,full HD).Finally,the mobile client decodes the received content and integrates the SI and original content,which produces a high-quality video.In our extensive simulation experiments,we observed that the amount of computation needed to construct a UHD video in the client is 50%-60% lower than that required to decode UHD video compressed by legacy video encoding algorithms.Moreover,the bandwidth required to transmit a full HD video and its side information is around 70% lower than that required for a normal UHD video.The subjective quality of the enhanced UHD is similar to that of the original UHD video even though the client pays lower communication costs with reduced computing power.

RTP-based broadcast streaming of high definition H.264/AVC video: an error robustness evaluation

In this work, we present an evaluation of the performance and error robustness of RTP-based broadcast streaming of high-quality high-definition (HD) H.264/AVC video. Using a fully controlled IP test bed (Hillestad et al., 2005), we broadcast high-definition video over RTP/UDP, and use an IP network emulator to introduce a varying amount of randomly distributed packet loss. A high-performance network interface monitoring card is used to capture the video packets into a trace file. Purpose-built software parses the trace file, analyzes the RTP stream and assembles the correctly received NAL units into an H.264/AVC Annex B byte stream file, which is subsequently decoded by JVT JM 10.1 reference software. The proposed measurement setup is a novel, practical and intuitive approach to perform error resilience testing of real-world H.264/AVC broadcast applications. Through a series of experiments, we evaluate some of the error resilience features of the H.264/AVC standard, and see how they perform at packet loss rates from 0.01% to 5%. The results confirmed that an appropriate slice partitioning scheme is essential to have a graceful degradation in received quality in the case of packet loss. While flexible macroblock ordering reduces the compression efficiency about 1 dB for our test material, reconstructed video quality is improved for loss rates above 0.25%.

Equation based rate control scheme for video streaming over wireless channels with link level ARQ

Equation based TCP-friendly rate control (TFRC) protocol has been proposed to support video streaming applications. In order to improve TFRC performance in wireless channels, the link level automatic repeat request (ARQ) scheme is usually deployed. However, ARQ cannot ensure strict delay guarantees, especially over multihop links. This paper introduces a theoretical model to deduce an equation for packet size adjustment in transport layer to minimize retransmission delay by taking into con- sideration the causative reasons inducing retransmission in link layer. An enhanced TFRC (ETFRC) scheme is proposed inte- grating TFRC with variable packet size policy. Simulation results demonstrate that higher goodput, lower packet loss rate (PLR), lower frame transmission delay and jitter with good fairness can be achieved by our proposed mechanism.

New Rate Allocation Method for MPEG-4 FGS Video Streaming

A new rate allocation method for fine-granular scalability (FGS) coded bitstreams is presented in order to achieve smooth quality reconstruction of frames under channel conditions with a wide range of bandwidth variation and improve the average PSNR of the whole sequence. Based on a quality weighted bit allocation method, a sliding window rate allocation method is proposed for the first time so that the window can slide along the video sequence with a certain sliding step. Experimental results show that, under dynamic bandwidth conditions, the proposed method can simultaneously satisfy the requirements for improving average PSNR of the whole video sequence greatly and reducing the fluctuations between adjacent frames greatly.

Unequal Forced-Intra-Refresh for robust video streaming

For video streaming over lossy channels, intra refresh can mitigate the error-propagation effect caused by packet losses Besides some intra-mode macroblocks （MBs） generated by the ＂Lagrangian rate-distortion＂ or ＂Sum of absolute difference＂ mode decision, the encoder or transcoder possibly needs to increase some ＂forced＂ intra-mode MBs for robust video streaming. Based on the error-propagation analysis in a group of pictures （GOP）, we propose an unequal Forced-Intra-Refresh （FIR） scheme to improve packet loss resilience of video streaming. According to a GOP-level error-propagation model, the proposed unequal FIR scheme can optimally increase the unequal number of forced intra-mode MBs for different frames in a GOP. Simulation results showed that the proposed scheme can effectively enhance the robustness of video streaming under different channel conditions, and achieve about 0. 1-0.9 dB gains over the average FIR scheme in H.264/AVC tools.

QUALITY-AWARE CROSS-LAYER SCHEDULING FOR ROBUST VIDEO STREAMING OVER WIRELESS CHANNELS

In the case of video streaming over wireless channels, burst errors may lead to serious video quality degradation. By jointly exploiting the scheduling mechanism on different communication layers, this paper proposes a quality-aware cross-layer scheduling scheme to achieve unequal error control for each Latency-constraint Frame Set (LFS) of a video stream. After a network-layer agent at base station firstly utilizes the network-layer packet scheduling to provide packet-granularity importance classifi-cation for the current LFS, a link-layer agent at base station further utilizes the Radio-Link-Unit (RLU) scheduling to implement finer selective retransmission of the current LFS. Under scheduling delay and bandwidth constraints, the proposed scheme can be aware of the application-layer quality and time-varying channel conditions, and hence burst errors can simply be shifted to lower-priority transmission units in the current LFS. Simulation results demonstrate that the proposed scheme has strong robustness against burst errors, and thus improves the overall received quality of the video stream over wireless channels.

Trade-off Optimization for Scalable Video Coding Streaming in Relay-Based OFDMA Networks

To achieve an optimal trade-off between video quality and energy efficiency in the uplink streaming of multi-user Scalable Video Coding (SVC) videos in relay-based Orthogonal Frequency Division Multiple Access (OFDMA) cellular networks, a cross-layer design framework that jointly selects the Transmission Policy (TP) for SVC video frames, assigns OFDMA subcarriers, and allocates power for each subcarrier is proposed. We apply the dual decomposition method to the problem, and obtain a TP selection subproblem for each SVC video adaptation and a resource allocation subproblem of Joint Subcarrier, Relay and Power Allocation (JSRPA). A second level of dual decomposition is used to divide the JSRPA problem into independent subcarrier subproblems. The proposed Crosslayer Trade-off Optimization (CTO) algorithm is sub-distributed with significantly low complexity. A performance evaluation with typical SVC video traces demonstrates that the proposed algorithm is able to converge and efficiently achieve the optimal trade-off between the video quality and energy consumption at the MSs for uplink SVC streaming.

Wireless ad hoc video transmission:a Bayesian network-based scheme

A novel bandwidth prediction and control scheme is proposed for video transmission over an ad boc network. The scheme is based on cross-layer, feedback, and Bayesian network techniques. The impacts of video quality are formulized and deduced. The relevant factors are obtained by a cross-layer mechanism or Feedback method. According to these relevant factors, the variable set and the Bayesian network topology are determined. Then a Bayesian network prediction model is constructed. The results of the prediction can be used as the bandwidth of the mobile ad hoc network （MANET）. According to the bandwidth, the video encoder is controlled to dynamically adjust and encode the right bit rates of a real-time video stream. Integrated simulation of a video streaming communication system is implemented to validate the proposed solution. In contrast to the conventional transfer scheme, the results of the experiment indicate that the proposed scheme can make the best use of the network bandwidth; there are considerable improvements in the packet loss and the visual quality of real-time video.K

Robust scalable pre-compressed video coding based on MPEG4-FGS over the Internet

Streaming video is becoming increasingly popular among Internet multimedia applications. A robust coding scheme for DCT-based scalable video streaming over the Internet is proposed in this paper. Compared with conventional MPEG4-FGS (fine granular scalable) and progressive FGS(PFGS), the proposed method generates the base layer including some sub-base layers by DCT coefficient reordering and VLC reshuffling, which enables the video stream of to adapt itself to long-term bandwidth time-varying of channel. Furthermore, a novel end-to-end transmission architecture for scalable video streaming over the Internet is also presented, in which an adaptive unequal packet loss protection (AUPLP) strategy is proposed to determine the currently available network bandwidth and adjust the sending rates according to different situations, such as network congestion or unreliable transmission. Experimental results show that the proposed progressive scalable scheme can improve the average coding efficiency up to 1.2 dB compared with MPEG4-FGS and PFGS in lower bandwidth, and the AUPLP strategy can improve the transmitting performances not only of the proposed scheme, but also of MPEG4-FGS, PFGS system.

A software-defined networking approach for handover management with real-time video in WLANs

To achieve high performance and reliability in video streaming over wireless local area networks （WLANs）, one must jointly consider both optimized association to access points （APs） and handover management based on dynamic scanning of alternate APs. In this article, we propose a new architecture within the software-defined networking （SDN） framework, which allows stations to be connected to several APs simultaneously and to switch fast between them. We evaluate our system in a real-time testbed and demonstrate that our SDN-based handover mechanism significantly reduces the number and duration of video freeze events and allows for smaller playout buffers.

QoE Assessment and Prediction Method for HighDefinition Video Stream Using Image Damage Accumulation

The accuracy of the traditional assessment method of the quality of experience(Qo E) has been facing challenges with the growth of high-definition(HD) video streaming services.Image display-quality damage is the main factor that affects the Qo E in HD video services through UDP network transmission.In this paper,we introduce a novel objective factor known as image damage accumulation(IDA) to assess user's Qo E in HD video services.First,this paper quantitatively analyzed the effect on user quality of experience by IDA and established a mapping relationship between mean opinion scores and IDA.Furthermore,the probability of image damage caused by compression and transmission were analyzed.Based on this analysis,an objective Qo E assessment and prediction method for HD video stream service that evaluated the user experience according to IDA are proposed.The proposed method can achieve assessment and prediction accuracy on three distinct subjective tests.

Encoder-Decoder Based LSTM Model to Advance User QoE in 360-Degree Video

The development of multimedia content has resulted in a massiveincrease in network traffic for video streaming. It demands such types ofsolutions that can be addressed to obtain the user’s Quality-of-Experience(QoE). 360-degree videos have already taken up the user’s behavior by storm.However, the users only focus on the part of 360-degree videos, known as aviewport. Despite the immense hype, 360-degree videos convey a loathsomeside effect about viewport prediction, making viewers feel uncomfortablebecause user viewport needs to be pre-fetched in advance. Ideally, we canminimize the bandwidth consumption if we know what the user motionin advance. Looking into the problem definition, we propose an EncoderDecoder based Long-Short Term Memory (LSTM) model to more accuratelycapture the non-linear relationship between past and future viewport positions. This model takes the transforming data instead of taking the direct inputto predict the future user movement. Then, this prediction model is combinedwith a rate adaptation approach that assigns the bitrates to various tiles for360-degree video frames under a given network capacity. Hence, our proposedwork aims to facilitate improved system performance when QoE parametersare jointly optimized. Some experiments were carried out and compared withexisting work to prove the performance of the proposed model. Last but notleast, the experiments implementation of our proposed work provides highuser’s QoE than its competitors.

Intelligent Mobile Video Surveillance System with Multilevel Distillation

This paper proposes a mobile video surveillance system consisting of intelligent video analysis and mobile communication networking. This multilevel distillation approach helps mobile users monitor tremendous surveillance videos on demand through video streaming over mobile communication networks. The intelligent video analysis includes moving object detection/tracking and key frame selection which can browse useful video clips. The communication networking services, comprising video transcoding, multimedia messaging, and mobile video streaming, transmit surveillance information into mobile appliances. Moving object detection is achieved by background subtraction and particle filter tracking. Key frame selection, which aims to deliver an alarm to a mobile client using multimedia messaging service accompanied with an extracted clear frame, is reached by devising a weighted importance criterion considering object clarity and face appearance. Besides, a spatial- domain cascaded transcoder is developed to convert the filtered image sequence of detected objects into the mobile video streaming format. Experimental results show that the system can successfully detect all events of moving objects for a complex surveillance scene, choose very appropriate key frames for users, and transcode the images with a high power signal-to-noise ratio （PSNR）.

Reversible Data Hiding Based on Run-Level Coding in H.264/AVC Video Streams

This paper presents a reversible data hiding(RDH)method,which is designed by combining histogram modification(HM)with run-level coding in H.264/advanced video coding(AVC).In this scheme,the run-level is changed for embedding data into H.264/AVC video sequences.In order to guarantee the reversibility of the proposed scheme,the last nonzero quantized discrete cosine transform(DCT)coefficients in embeddable 4×4 blocks are shifted by the technology of histogram modification.The proposed scheme is realized after quantization and before entropy coding of H.264/AVC compression standard.Therefore,the embedded information can be correctly extracted at the decoding side.Peak-signal-noise-to-ratio(PSNR)and Structure similarity index(SSIM),embedding payload and bit-rate variation are exploited to measure the performance of the proposed scheme.Experimental results have shown that the proposed scheme leads to less SSIM variation and bit-rate increase.