Workout Action Recognition in Video Streams Using an Attention Driven Residual DC-GRU Network

Regular exercise is a crucial aspect of daily life, as it enables individuals to stay physically active, lowers thelikelihood of developing illnesses, and enhances life expectancy. The recognition of workout actions in videostreams holds significant importance in computer vision research, as it aims to enhance exercise adherence, enableinstant recognition, advance fitness tracking technologies, and optimize fitness routines. However, existing actiondatasets often lack diversity and specificity for workout actions, hindering the development of accurate recognitionmodels. To address this gap, the Workout Action Video dataset (WAVd) has been introduced as a significantcontribution. WAVd comprises a diverse collection of labeled workout action videos, meticulously curated toencompass various exercises performed by numerous individuals in different settings. This research proposes aninnovative framework based on the Attention driven Residual Deep Convolutional-Gated Recurrent Unit (ResDCGRU)network for workout action recognition in video streams. Unlike image-based action recognition, videoscontain spatio-temporal information, making the task more complex and challenging. While substantial progresshas been made in this area, challenges persist in detecting subtle and complex actions, handling occlusions,and managing the computational demands of deep learning approaches. The proposed ResDC-GRU Attentionmodel demonstrated exceptional classification performance with 95.81% accuracy in classifying workout actionvideos and also outperformed various state-of-the-art models. The method also yielded 81.6%, 97.2%, 95.6%, and93.2% accuracy on established benchmark datasets, namely HMDB51, Youtube Actions, UCF50, and UCF101,respectively, showcasing its superiority and robustness in action recognition. The findings suggest practicalimplications in real-world scenarios where precise video action recognition is paramount, addressing the persistingchallenges in the field. TheWAVd dataset serves as a catalyst for the development ofmore robust and effective fitnesstracking systems and ultimately promotes healthier lifestyles through improved exercise monitoring and analysis.

Machine Learning Based Classifiers for QoE Prediction Framework in Video Streaming over 5G Wireless Networks

Recently,the combination of video services and 5G networks have been gaining attention in the wireless communication realm.With the brisk advancement in 5G network usage and the massive popularity of threedimensional video streaming,the quality of experience(QoE)of video in 5G systems has been receiving overwhelming significance from both customers and service provider ends.Therefore,effectively categorizing QoE-aware video streaming is imperative for achieving greater client satisfaction.This work makes the following contribution:First,a simulation platform based on NS-3 is introduced to analyze and improve the performance of video services.The simulation is formulated to offer real-time measurements,saving the expensive expenses associated with real-world equipment.Second,A valuable framework for QoE-aware video streaming categorization is introduced in 5G networks based on machine learning(ML)by incorporating the hyperparameter tuning(HPT)principle.It implements an enhanced hyperparameter tuning(EHPT)ensemble and decision tree(DT)classifier for video streaming categorization.The performance of the ML approach is assessed by considering precision,accuracy,recall,and computation time metrics for manifesting the superiority of these classifiers regarding video streaming categorization.This paper demonstrates that our ML classifiers achieve QoE prediction accuracy of 92.59%for(EHPT)ensemble and 87.037%for decision tree(DT)classifiers.

Adaptive Learning Video Streaming with QoE in Multi-Home Heterogeneous Networks

In recent years,real-time video streaming has grown in popularity.The growing popularity of the Internet of Things(IoT)and other wireless heterogeneous networks mandates that network resources be carefully apportioned among versatile users in order to achieve the best Quality of Experience(QoE)and performance objectives.Most researchers focused on Forward Error Correction(FEC)techniques when attempting to strike a balance between QoE and performance.However,as network capacity increases,the performance degrades,impacting the live visual experience.Recently,Deep Learning(DL)algorithms have been successfully integrated with FEC to stream videos across multiple heterogeneous networks.But these algorithms need to be changed to make the experience better without sacrificing packet loss and delay time.To address the previous challenge,this paper proposes a novel intelligent algorithm that streams video in multi-home heterogeneous networks based on network-centric characteristics.The proposed framework contains modules such as Intelligent Content Extraction Module(ICEM),Channel Status Monitor(CSM),and Adaptive FEC(AFEC).This framework adopts the Cognitive Learning-based Scheduling(CLS)Module,which works on the deep Reinforced Gated Recurrent Networks(RGRN)principle and embeds them along with the FEC to achieve better performances.The complete framework was developed using the Objective Modular Network Testbed in C++(OMNET++),Internet networking(INET),and Python 3.10,with Keras as the front end and Tensorflow 2.10 as the back end.With extensive experimentation,the proposed model outperforms the other existing intelligentmodels in terms of improving the QoE,minimizing the End-to-End Delay(EED),and maintaining the highest accuracy(98%)and a lower Root Mean Square Error(RMSE)value of 0.001.

MEC-Assisted Flexible Transcoding Strategy for Adaptive Bitrate Video Streaming in Small Cell Networks

Adaptive bitrate video streaming(ABR)has become a critical technique for mobile video streaming to cope with time-varying network conditions and different user preferences.However,there are still many problems in achieving high-quality ABR video streaming over cellular networks.Mobile Edge Computing(MEC)is a promising paradigm to overcome the above problems by providing video transcoding capability and caching the ABR video streaming within the radio access network(RAN).In this paper,we propose a flexible transcoding strategy to provide viewers with low-latency video streaming services in the MEC networks under the limited storage,computing,and spectrum resources.According to the information collected from users,the MEC server acts as a controlling component to adjust the transcoding strategy flexibly based on optimizing the video caching placement strategy.Specifically,we cache the proper bitrate version of the video segments at the edge servers and select the appropriate bitrate version of the video segments to perform transcoding under jointly considering access control,resource allocation,and user preferences.We formulate this problem as a nonconvex optimization and mixed combinatorial problem.Moreover,the simulation results indicate that our proposed algorithm can ensure a low-latency viewing experience for users.

Delay-Based Cross-Layer QoS Scheme for Video Streaming in Wireless Ad Hoc Networks

With correlating with human perception, quality of experience(Qo E) is also an important measurement in evaluation of video quality in addition to quality of service(Qo S). A cross-layer scheme based on Lyapunov optimization framework for H.264/AVC video streaming over wireless Ad hoc networks is proposed, with increasing both Qo E and Qo S performances. Different from existing works, this scheme routes and schedules video packets according to the statuses of the frame buffers at the destination nodes to reduce buffer underflows and to increase video playout continuity. The waiting time of head-ofline packets of data queues are considered in routing and scheduling to reduce the average end-to-end delay of video sessions. Different types of packets are allocated with different priorities according to their generated rates under H.264/AVC. To reduce the computational complexity, a distributed media access control policy and a power control algorithm cooperating with the media access policy are proposed. Simulation results show that, compared with existing schemes, this scheme can improve both the Qo S and Qo E performances. The average peak signal-to-noise ratio(PSNR) of the received video streams is also increased.

Efficient Computation Offloading in Mobile Cloud Computing for Video Streaming Over 5G

In this paper,we investigate video quality enhancement using computation offloading to the mobile cloud computing(MCC)environment.Our objective is to reduce the computational complexity required to covert a low-resolution video to high-resolution video while minimizing computation at the mobile client and additional communication costs.To do so,we propose an energy-efficient computation offloading framework for video streaming services in a MCC over the fifth generation(5G)cellular networks.In the proposed framework,the mobile client offloads the computational burden for the video enhancement to the cloud,which renders the side information needed to enhance video without requiring much computation by the client.The cloud detects edges from the upsampled ultra-high-resolution video(UHD)and then compresses and transmits them as side information with the original low-resolution video(e.g.,full HD).Finally,the mobile client decodes the received content and integrates the SI and original content,which produces a high-quality video.In our extensive simulation experiments,we observed that the amount of computation needed to construct a UHD video in the client is 50%-60% lower than that required to decode UHD video compressed by legacy video encoding algorithms.Moreover,the bandwidth required to transmit a full HD video and its side information is around 70% lower than that required for a normal UHD video.The subjective quality of the enhanced UHD is similar to that of the original UHD video even though the client pays lower communication costs with reduced computing power.

QoE Assessment and Prediction Method for HighDefinition Video Stream Using Image Damage Accumulation

The accuracy of the traditional assessment method of the quality of experience(Qo E) has been facing challenges with the growth of high-definition(HD) video streaming services.Image display-quality damage is the main factor that affects the Qo E in HD video services through UDP network transmission.In this paper,we introduce a novel objective factor known as image damage accumulation(IDA) to assess user's Qo E in HD video services.First,this paper quantitatively analyzed the effect on user quality of experience by IDA and established a mapping relationship between mean opinion scores and IDA.Furthermore,the probability of image damage caused by compression and transmission were analyzed.Based on this analysis,an objective Qo E assessment and prediction method for HD video stream service that evaluated the user experience according to IDA are proposed.The proposed method can achieve assessment and prediction accuracy on three distinct subjective tests.

New Rate Allocation Method for MPEG-4 FGS Video Streaming

A new rate allocation method for fine-granular scalability (FGS) coded bitstreams is presented in order to achieve smooth quality reconstruction of frames under channel conditions with a wide range of bandwidth variation and improve the average PSNR of the whole sequence. Based on a quality weighted bit allocation method, a sliding window rate allocation method is proposed for the first time so that the window can slide along the video sequence with a certain sliding step. Experimental results show that, under dynamic bandwidth conditions, the proposed method can simultaneously satisfy the requirements for improving average PSNR of the whole video sequence greatly and reducing the fluctuations between adjacent frames greatly.

Equation based rate control scheme for video streaming over wireless channels with link level ARQ

Equation based TCP-friendly rate control (TFRC) protocol has been proposed to support video streaming applications. In order to improve TFRC performance in wireless channels, the link level automatic repeat request (ARQ) scheme is usually deployed. However, ARQ cannot ensure strict delay guarantees, especially over multihop links. This paper introduces a theoretical model to deduce an equation for packet size adjustment in transport layer to minimize retransmission delay by taking into con- sideration the causative reasons inducing retransmission in link layer. An enhanced TFRC (ETFRC) scheme is proposed inte- grating TFRC with variable packet size policy. Simulation results demonstrate that higher goodput, lower packet loss rate (PLR), lower frame transmission delay and jitter with good fairness can be achieved by our proposed mechanism.

Unequal Forced-Intra-Refresh for robust video streaming

For video streaming over lossy channels, intra refresh can mitigate the error-propagation effect caused by packet losses Besides some intra-mode macroblocks （MBs） generated by the ＂Lagrangian rate-distortion＂ or ＂Sum of absolute difference＂ mode decision, the encoder or transcoder possibly needs to increase some ＂forced＂ intra-mode MBs for robust video streaming. Based on the error-propagation analysis in a group of pictures （GOP）, we propose an unequal Forced-Intra-Refresh （FIR） scheme to improve packet loss resilience of video streaming. According to a GOP-level error-propagation model, the proposed unequal FIR scheme can optimally increase the unequal number of forced intra-mode MBs for different frames in a GOP. Simulation results showed that the proposed scheme can effectively enhance the robustness of video streaming under different channel conditions, and achieve about 0. 1-0.9 dB gains over the average FIR scheme in H.264/AVC tools.

QUALITY-AWARE CROSS-LAYER SCHEDULING FOR ROBUST VIDEO STREAMING OVER WIRELESS CHANNELS

In the case of video streaming over wireless channels, burst errors may lead to serious video quality degradation. By jointly exploiting the scheduling mechanism on different communication layers, this paper proposes a quality-aware cross-layer scheduling scheme to achieve unequal error control for each Latency-constraint Frame Set (LFS) of a video stream. After a network-layer agent at base station firstly utilizes the network-layer packet scheduling to provide packet-granularity importance classifi-cation for the current LFS, a link-layer agent at base station further utilizes the Radio-Link-Unit (RLU) scheduling to implement finer selective retransmission of the current LFS. Under scheduling delay and bandwidth constraints, the proposed scheme can be aware of the application-layer quality and time-varying channel conditions, and hence burst errors can simply be shifted to lower-priority transmission units in the current LFS. Simulation results demonstrate that the proposed scheme has strong robustness against burst errors, and thus improves the overall received quality of the video stream over wireless channels.

Reversible Data Hiding Based on Run-Level Coding in H.264/AVC Video Streams

This paper presents a reversible data hiding(RDH)method,which is designed by combining histogram modification(HM)with run-level coding in H.264/advanced video coding(AVC).In this scheme,the run-level is changed for embedding data into H.264/AVC video sequences.In order to guarantee the reversibility of the proposed scheme,the last nonzero quantized discrete cosine transform(DCT)coefficients in embeddable 4×4 blocks are shifted by the technology of histogram modification.The proposed scheme is realized after quantization and before entropy coding of H.264/AVC compression standard.Therefore,the embedded information can be correctly extracted at the decoding side.Peak-signal-noise-to-ratio(PSNR)and Structure similarity index(SSIM),embedding payload and bit-rate variation are exploited to measure the performance of the proposed scheme.Experimental results have shown that the proposed scheme leads to less SSIM variation and bit-rate increase.

Smart Instrumentation in Video Streams Using Real-time Bit-rate Adaptation

With the proliferation of video traffic across the Internet and wireless networks,various compression standards for videos have emerged over the past two decades.Among them,Motion Joint Photographic Expects Group(M-JPEG)offers the advantages of no frame-to-frame error propagation,less computation cost,and achieving a short latency in both encoding and decoding.However,the bit-rate of M-JPEG stream is variable due to its dynamic frame size,and that leads to adverse outcomes such as inducing different quality-of-service(QoS)grades from servers and networks and inducing disturbances in a real-time network environment.This paper proposes a novel approach that can control bit-rate and also the individual frame size of M-JPEG video stream in real-time.Experimental results are provided to show that the proposed approach is amenable to direct,straightforward implementation and yet outperforms similar existing approaches in regulating the bit-rate and the frame size of M-JPEG streams.

Error resilient concurrent video streaming over wireless mesh networks

In this paper, we propose a multi-source multi-path video streaming system for supporting high quality concurrent video-on-demand (VoD) services over wireless mesh networks (WMNs), and leverage forward error correction to enhance the error resilience of the system. By taking wireless interference into consideration, we present a more realistic networking model to capture the characteristics of WMNs and then design a route selection scheme using a joint rate/interference-distortion optimiza- tion framework to help the system optimally select concurrent streaming paths. We mathematically formulate such a route selec- tion problem, and solve it heuristically using genetic algorithm. Simulation results demonstrate the effectiveness of our proposed scheme.

Quality Assessment and Measurement for Internet Video Streaming

Benefiting from the improvements of Internet infrastructure and video coding technology, online video services are becoming a new favorite form of video entertainment.However, most of the existing video quality assessment methods are designed for broadcasting/cable televisions and it is still an open issue how to assess and measure the quality of online video services. In this paper, we survey the state-of-the-art video streaming technologies, and present a framework of quality assessment and measurement for Internet video streaming. This paper introduces several metrics for user's quality of experience(QoE).These QoE metrics are classified into two categories: objective metrics and subjective metrics. It is different for service participators to measure objective and subjective metrics.The QoE measurement methodologies consist of client-side, server-side, and in-network measurement.

On optimal receiver buffer size in adaptive Internet video streaming

The effect of receiver buffer size on perceived video quality of an Internet video streamer application was examined in this work. Several network conditions and several versions of the application are used to gain understanding of the response to varying buffer sizes. Among these conditions local area versus wide area, bandwidth estimation based versus non-bandwidth estimation based cases are examined in detail. A total of 1000 min of video is streamed over Intemet and statistics are collected. It was observed that when bandwidth estimation is possible, choosing larger buffer size for higher available bandwidth yields quality increase in perceived video.

Network-adaptive HD MPEG-2 video streaming with cross-layered channel monitoring in WLAN

In this paper, we propose a practical design and implementation of network-adaptive high definition (HD) MPEG-2 video streaming combined with cross-layered channel monitoring (CLM) over the IEEE 802.11a wireless local area network (WLAN). For wireless channel monitoring, we adopt a cross-layered approach, where an access point (AP) periodically measures lower layers such as medium access control (MAC) and physical (PHY) transmission information (e.g., MAC layer loss rate) and then sends the monitored information to the streaming server application. The adaptive streaming server with the CLM scheme reacts more quickly and efficiently to the fluctuating wireless channel than the end-to-end application-layer monitoring (E2EM) scheme. The streaming server dynamically performs priority-based frame dropping to adjust the sending rate according to the measured wireless channel condition. For this purpose, the proposed streaming system nicely provides frame-based prioritized packetization by using a real-time stream parsing module. Various evaluation results over an IEEE 802.11a WLAN testbed are provided to verify the intended Quality of Service (QoS) adaptation capability. Experimental results showed that the proposed system can mitigate the quality degradation of video streaming due to the fluctuations of time-varying channel.

Joint rate control and scheduling for wireless uplink video streaming

We solve the problem of uplink video streaming in CDMA cellular networks by jointly designing the rate control and scheduling algorithms. In the pricing-based distributed rate control algorithm, the base station announces a price for the per unit average rate it can support, and the mobile devices choose their desired average transmission rates by balancing their video quality and cost of transmission. Each mobile device then determines the specific video frames to transmit by a video summarization process. In the time-division-multiplexing (TDM) scheduling algorithm, the base station collects the information on frames to be transmitted from all devices within the current time window, sorts them in increasing order of deadlines, and schedules the transmissions in a TDM fashion. This joint algorithm takes advantage of the multi-user content diversity, and maximizes the network total utility (i.e., minimize the network total distortion), while satisfying the delivery deadline constraints. Simulations showed that the proposed algorithm significantly outperforms the constant rate provision algorithm.

Joint routing and rate allocation for multiple video streams in ad-hoc wireless networks

The support for multiple video streams in an ad-hoc wireless network requires appropriate routing and rate allocation measures ascertaining the set of links for transmitting each stream and the encoding rate of the video to be delivered over the chosen links. The routing and rate allocation procedures impact the sustained quality of each video stream measured as the mean squared error (MSE) distortion at the receiver, and the overall network congestion in terms of queuing delay per link. We study the trade-off between these two competing objectives in a convex optimization formulation, and discuss both centralized and dis- tributed solutions for joint routing and rate allocation for multiple streams. For each stream, the optimal allocated rate strikes a balance between the selfish motive of minimizing video distortion and the global good of minimizing network congestions, while the routes are chosen over the least-congested links in the network. In addition to detailed analysis, network simulation results using ns-2 are presented for studying the optimal choice of parameters and to confirm the effectiveness of the proposed measures.

Content-adaptive robust error concealment for packet-lossy H.264 video streaming

In this paper, we present a spatio-temporal post-processing error concealment (EC) algorithm designed initially for a H.264 video-streaming scheme over packet-lossy networks. It aims at optimizing the subjective quality of the restored video under the constraints of low delay and computational complexity, which are critical to real-time applications and portable devices having limited resources. Specifically, it takes into consideration the physical property of motion field in order to achieve more meaningful perceptual video quality, in addition to the improved objective PSNR. Further, a simple bilinear spatial interpolation approach is combined with the improved boundary-match (B-M) based temporal EC approach according to texture and motion activity analysis. Finally, we propose a low complexity temporal EC method based on motion vector interpolation as a replacement of the B-M based approach in the scheme under low-computation requirement, or as a complement to further improve the scheme's performance in applications having enough computation resources. Extensive experiments demonstrated that the proposal features not only better reconstruction, objectively and subjectively, than JM benchmark, but also robustness to different video sequences.