Workout Action Recognition in Video Streams Using an Attention Driven Residual DC-GRU Network

Regular exercise is a crucial aspect of daily life, as it enables individuals to stay physically active, lowers thelikelihood of developing illnesses, and enhances life expectancy. The recognition of workout actions in videostreams holds significant importance in computer vision research, as it aims to enhance exercise adherence, enableinstant recognition, advance fitness tracking technologies, and optimize fitness routines. However, existing actiondatasets often lack diversity and specificity for workout actions, hindering the development of accurate recognitionmodels. To address this gap, the Workout Action Video dataset (WAVd) has been introduced as a significantcontribution. WAVd comprises a diverse collection of labeled workout action videos, meticulously curated toencompass various exercises performed by numerous individuals in different settings. This research proposes aninnovative framework based on the Attention driven Residual Deep Convolutional-Gated Recurrent Unit (ResDCGRU)network for workout action recognition in video streams. Unlike image-based action recognition, videoscontain spatio-temporal information, making the task more complex and challenging. While substantial progresshas been made in this area, challenges persist in detecting subtle and complex actions, handling occlusions,and managing the computational demands of deep learning approaches. The proposed ResDC-GRU Attentionmodel demonstrated exceptional classification performance with 95.81% accuracy in classifying workout actionvideos and also outperformed various state-of-the-art models. The method also yielded 81.6%, 97.2%, 95.6%, and93.2% accuracy on established benchmark datasets, namely HMDB51, Youtube Actions, UCF50, and UCF101,respectively, showcasing its superiority and robustness in action recognition. The findings suggest practicalimplications in real-world scenarios where precise video action recognition is paramount, addressing the persistingchallenges in the field. TheWAVd dataset serves as a catalyst for the development ofmore robust and effective fitnesstracking systems and ultimately promotes healthier lifestyles through improved exercise monitoring and analysis.

VB环境下Audio/Video压缩数据流播放技术的应用

介绍了在 Windwos操作系统中 ,利用 VB自身条件和原有控件 ,灵活调用 Windows下的动态链接库开发

Evaluation of packet loss impairment on streaming video

Video compression technologies are essential in video streaming application because they could save a great amount of network resources. However compressed videos are also extremely sensitive to packet loss which is inevitable in today's best effort IP network. Therefore we think accurate evaluation of packet loss impairment on compressed video is very important. In this work, we develop an analytic model to describe these impairments without the reference of the original video (NR) and propose an impairment metric based on the model, which takes into account both impairment length and impairment strength. To evaluate an impaired frame or video, we design a detection and evaluation algorithm (DE algorithm) to compute the above metric value. The DE algorithm has low computational complexity and is currently being implemented in the real-time monitoring module of our HDTV over IP system. The impairment metric and DE algorithm could also be used in adaptive system or be used to compare diffeient error concealment strategies.

Error resilient concurrent video streaming over wireless mesh networks

In this paper, we propose a multi-source multi-path video streaming system for supporting high quality concurrent video-on-demand (VoD) services over wireless mesh networks (WMNs), and leverage forward error correction to enhance the error resilience of the system. By taking wireless interference into consideration, we present a more realistic networking model to capture the characteristics of WMNs and then design a route selection scheme using a joint rate/interference-distortion optimiza- tion framework to help the system optimally select concurrent streaming paths. We mathematically formulate such a route selec- tion problem, and solve it heuristically using genetic algorithm. Simulation results demonstrate the effectiveness of our proposed scheme.

Video transcoding for adaptive bitrate streaming over edge-cloud continuum

Video transcoding is to create multiple representations of a video for content adaptation.It is deemed as a core technique in Adaptive BitRate(ABR)streaming.How to manage video transcoding affects the performance of ABR streaming in various aspects,including operational cost,streaming delays,Quality of Experience(QoE),etc.Therefore,the problems of implementing video transcoding in ABR streaming must be systematically studied to improve the overall performance of the streaming services.These problems become more worthy of investigation with the emergence of the edge-cloud continuum,which makes the resource allocation for video transcoding more complicated.To this end,this paper provides an investigation of the main technical problems related to video transcoding in ABR streaming,including designing a rate profile for video transcoding,providing resources for video transcoding in clouds,and caching multi-bitrate video contents in networks,etc.We analyze these problems from the perspective of resource allocation in the edge-cloud continuum and cast them into resource and Quality of Service(QoS)optimization problems.The goal is to minimize resource consumption while guaranteeing the QoS for ABR streaming.We also discuss some promising research directions for the ABR streaming services.

Rate-distortion optimized frame dropping and scheduling for multi-user conversational and streaming video

We propose a Rate-Distortion (RD) optimized strategy for frame-dropping and scheduling of multi-user conversa- tional and streaming videos. We consider a scenario where conversational and streaming videos share the forwarding resources at a network node. Two buffers are setup on the node to temporarily store the packets for these two types of video applications. For streaming video, a big buffer is used as the associated delay constraint of the application is moderate and a very small buffer is used for conversational video to ensure that the forwarding delay of every packet is limited. A scheduler is located behind these two buffers that dynamically assigns transmission slots on the outgoing link to the two buffers. Rate-distortion side information is used to perform RD-optimized frame dropping in case of node overload. Sharing the data rate on the outgoing link between the con- versational and the streaming videos is done either based on the fullness of the two associated buffers or on the mean incoming rates of the respective videos. Simulation results showed that our proposed RD-optimized frame dropping and scheduling ap- proach provides significant improvements in performance over the popular priority-based random dropping (PRD) technique.

Optimized viewport-adaptive 360-degree video streaming

Because of the rapid growth of head-mounted displays and 5G networking deployment,360-degree video has become increasingly popular.To generate the real experience of a virtual environment,360-degree videos require an ultrahigh resolution and frame rate to cover an omnidirectional view.These two prerequisites impose challenges for the transmission bandwidth and storage capacity of 360-degree video streaming.To reduce bandwidth and storage waste while providing a good immersive experience,we propose an optimized viewport-adaptive 360-degree video streaming method using high-efficiency video coding tiling,motion-constrained tile sets and MPEG dynamic adaptive streaming over HTTP spatial relationship description.The paper describes the rigorous design of the optimized system,which can assign different bitrates to different tiles in the viewport.The experimental results show that the proposed streaming system compares favourably with existing methods in terms of bitrate savings and storage capacity reduction.

CA-AODV: Congestion Adaptive AODV Routing Protocol for Streaming Video in Mobile Ad Hoc Networks

Whenever streaming of multimedia based data such as video, audio and text is performed traffic will be more and network becomes congested in mobile ad hoc networks. The present routing protocols are not able to cope up with this situation. It is observed that network congestion is the dominant reason for packet loss, longer delay and delay jitter in streaming video. Most of the present routing protocols are not designed to adapt to congestion control. We propose a new routing protocol, Congestion Adaptive AODV Routing Protocol (CA-AODV), to address the congestion issues considering delay, packet loss and routing overhead. To evaluate their performance, we have considered mpeg4 for streaming video data using network simulator (NS2). CA-AODV outperforms present protocols in delivery ratio and delay, while introducing less routing protocol overhead. The result demonstrates that integrating congestion adaptive mechanisms with AODV is a promising way to improve performance for heavy traffic load in multimedia based mobile ad hoc networks.

Quality Assessment and Measurement for Internet Video Streaming

Benefiting from the improvements of Internet infrastructure and video coding technology, online video services are becoming a new favorite form of video entertainment.However, most of the existing video quality assessment methods are designed for broadcasting/cable televisions and it is still an open issue how to assess and measure the quality of online video services. In this paper, we survey the state-of-the-art video streaming technologies, and present a framework of quality assessment and measurement for Internet video streaming. This paper introduces several metrics for user's quality of experience(QoE).These QoE metrics are classified into two categories: objective metrics and subjective metrics. It is different for service participators to measure objective and subjective metrics.The QoE measurement methodologies consist of client-side, server-side, and in-network measurement.

RTP-based broadcast streaming of high definition H.264/AVC video: an error robustness evaluation

In this work, we present an evaluation of the performance and error robustness of RTP-based broadcast streaming of high-quality high-definition (HD) H.264/AVC video. Using a fully controlled IP test bed (Hillestad et al., 2005), we broadcast high-definition video over RTP/UDP, and use an IP network emulator to introduce a varying amount of randomly distributed packet loss. A high-performance network interface monitoring card is used to capture the video packets into a trace file. Purpose-built software parses the trace file, analyzes the RTP stream and assembles the correctly received NAL units into an H.264/AVC Annex B byte stream file, which is subsequently decoded by JVT JM 10.1 reference software. The proposed measurement setup is a novel, practical and intuitive approach to perform error resilience testing of real-world H.264/AVC broadcast applications. Through a series of experiments, we evaluate some of the error resilience features of the H.264/AVC standard, and see how they perform at packet loss rates from 0.01% to 5%. The results confirmed that an appropriate slice partitioning scheme is essential to have a graceful degradation in received quality in the case of packet loss. While flexible macroblock ordering reduces the compression efficiency about 1 dB for our test material, reconstructed video quality is improved for loss rates above 0.25%.

MEC-Assisted Flexible Transcoding Strategy for Adaptive Bitrate Video Streaming in Small Cell Networks

Adaptive bitrate video streaming(ABR)has become a critical technique for mobile video streaming to cope with time-varying network conditions and different user preferences.However,there are still many problems in achieving high-quality ABR video streaming over cellular networks.Mobile Edge Computing(MEC)is a promising paradigm to overcome the above problems by providing video transcoding capability and caching the ABR video streaming within the radio access network(RAN).In this paper,we propose a flexible transcoding strategy to provide viewers with low-latency video streaming services in the MEC networks under the limited storage,computing,and spectrum resources.According to the information collected from users,the MEC server acts as a controlling component to adjust the transcoding strategy flexibly based on optimizing the video caching placement strategy.Specifically,we cache the proper bitrate version of the video segments at the edge servers and select the appropriate bitrate version of the video segments to perform transcoding under jointly considering access control,resource allocation,and user preferences.We formulate this problem as a nonconvex optimization and mixed combinatorial problem.Moreover,the simulation results indicate that our proposed algorithm can ensure a low-latency viewing experience for users.

Real-time Audio &Video Transmission System Based on Visible Light Communication

With the increasing popularity of solid sate lighting devices, Visible Light Communication (VLC) is globally recognized as an advanced and promising technology to realize short-range, high speed as well as large capacity wireless data transmission. In this paper, we propose a prototype of real-time audio and video broadcast system using inexpensive commercially available light emitting diode (LED) lamps. Experimental results show that real-time high quality audio and video with the maximum distance of 3 m can be achieved through proper layout of LED sources and improvement of concentration effects. Lighting model within room environment is designed and simulated which indicates close relationship between layout of light sources and distribution of illuminance.

Delay-Based Cross-Layer QoS Scheme for Video Streaming in Wireless Ad Hoc Networks

With correlating with human perception, quality of experience(Qo E) is also an important measurement in evaluation of video quality in addition to quality of service(Qo S). A cross-layer scheme based on Lyapunov optimization framework for H.264/AVC video streaming over wireless Ad hoc networks is proposed, with increasing both Qo E and Qo S performances. Different from existing works, this scheme routes and schedules video packets according to the statuses of the frame buffers at the destination nodes to reduce buffer underflows and to increase video playout continuity. The waiting time of head-ofline packets of data queues are considered in routing and scheduling to reduce the average end-to-end delay of video sessions. Different types of packets are allocated with different priorities according to their generated rates under H.264/AVC. To reduce the computational complexity, a distributed media access control policy and a power control algorithm cooperating with the media access policy are proposed. Simulation results show that, compared with existing schemes, this scheme can improve both the Qo S and Qo E performances. The average peak signal-to-noise ratio(PSNR) of the received video streams is also increased.

Efficient Computation Offloading in Mobile Cloud Computing for Video Streaming Over 5G

In this paper,we investigate video quality enhancement using computation offloading to the mobile cloud computing(MCC)environment.Our objective is to reduce the computational complexity required to covert a low-resolution video to high-resolution video while minimizing computation at the mobile client and additional communication costs.To do so,we propose an energy-efficient computation offloading framework for video streaming services in a MCC over the fifth generation(5G)cellular networks.In the proposed framework,the mobile client offloads the computational burden for the video enhancement to the cloud,which renders the side information needed to enhance video without requiring much computation by the client.The cloud detects edges from the upsampled ultra-high-resolution video(UHD)and then compresses and transmits them as side information with the original low-resolution video(e.g.,full HD).Finally,the mobile client decodes the received content and integrates the SI and original content,which produces a high-quality video.In our extensive simulation experiments,we observed that the amount of computation needed to construct a UHD video in the client is 50%-60% lower than that required to decode UHD video compressed by legacy video encoding algorithms.Moreover,the bandwidth required to transmit a full HD video and its side information is around 70% lower than that required for a normal UHD video.The subjective quality of the enhanced UHD is similar to that of the original UHD video even though the client pays lower communication costs with reduced computing power.

Machine Learning Based Classifiers for QoE Prediction Framework in Video Streaming over 5G Wireless Networks

Recently,the combination of video services and 5G networks have been gaining attention in the wireless communication realm.With the brisk advancement in 5G network usage and the massive popularity of threedimensional video streaming,the quality of experience(QoE)of video in 5G systems has been receiving overwhelming significance from both customers and service provider ends.Therefore,effectively categorizing QoE-aware video streaming is imperative for achieving greater client satisfaction.This work makes the following contribution:First,a simulation platform based on NS-3 is introduced to analyze and improve the performance of video services.The simulation is formulated to offer real-time measurements,saving the expensive expenses associated with real-world equipment.Second,A valuable framework for QoE-aware video streaming categorization is introduced in 5G networks based on machine learning(ML)by incorporating the hyperparameter tuning(HPT)principle.It implements an enhanced hyperparameter tuning(EHPT)ensemble and decision tree(DT)classifier for video streaming categorization.The performance of the ML approach is assessed by considering precision,accuracy,recall,and computation time metrics for manifesting the superiority of these classifiers regarding video streaming categorization.This paper demonstrates that our ML classifiers achieve QoE prediction accuracy of 92.59%for(EHPT)ensemble and 87.037%for decision tree(DT)classifiers.

QoE Assessment and Prediction Method for HighDefinition Video Stream Using Image Damage Accumulation

The accuracy of the traditional assessment method of the quality of experience(Qo E) has been facing challenges with the growth of high-definition(HD) video streaming services.Image display-quality damage is the main factor that affects the Qo E in HD video services through UDP network transmission.In this paper,we introduce a novel objective factor known as image damage accumulation(IDA) to assess user's Qo E in HD video services.First,this paper quantitatively analyzed the effect on user quality of experience by IDA and established a mapping relationship between mean opinion scores and IDA.Furthermore,the probability of image damage caused by compression and transmission were analyzed.Based on this analysis,an objective Qo E assessment and prediction method for HD video stream service that evaluated the user experience according to IDA are proposed.The proposed method can achieve assessment and prediction accuracy on three distinct subjective tests.

Integrating Audio-Visual Features and Text Information for Story Segmentation of News Video

Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The proposed approach detects the topic-caption frames, and integrates them with silence clips detection results, as well as shot segmentation results to locate the news story boundaries. The integration of audio-visual features and text information overcomes the weakness of the approach using only image analysis techniques. On test data with 135 400 frames, when the boundaries between news stories are detected, the accuracy rate 85.8% and the recall rate 97.5% are obtained. The experimental results show the approach is valid and robust.

Network-adaptive HD MPEG-2 video streaming with cross-layered channel monitoring in WLAN

In this paper, we propose a practical design and implementation of network-adaptive high definition (HD) MPEG-2 video streaming combined with cross-layered channel monitoring (CLM) over the IEEE 802.11a wireless local area network (WLAN). For wireless channel monitoring, we adopt a cross-layered approach, where an access point (AP) periodically measures lower layers such as medium access control (MAC) and physical (PHY) transmission information (e.g., MAC layer loss rate) and then sends the monitored information to the streaming server application. The adaptive streaming server with the CLM scheme reacts more quickly and efficiently to the fluctuating wireless channel than the end-to-end application-layer monitoring (E2EM) scheme. The streaming server dynamically performs priority-based frame dropping to adjust the sending rate according to the measured wireless channel condition. For this purpose, the proposed streaming system nicely provides frame-based prioritized packetization by using a real-time stream parsing module. Various evaluation results over an IEEE 802.11a WLAN testbed are provided to verify the intended Quality of Service (QoS) adaptation capability. Experimental results showed that the proposed system can mitigate the quality degradation of video streaming due to the fluctuations of time-varying channel.

Joint rate control and scheduling for wireless uplink video streaming

We solve the problem of uplink video streaming in CDMA cellular networks by jointly designing the rate control and scheduling algorithms. In the pricing-based distributed rate control algorithm, the base station announces a price for the per unit average rate it can support, and the mobile devices choose their desired average transmission rates by balancing their video quality and cost of transmission. Each mobile device then determines the specific video frames to transmit by a video summarization process. In the time-division-multiplexing (TDM) scheduling algorithm, the base station collects the information on frames to be transmitted from all devices within the current time window, sorts them in increasing order of deadlines, and schedules the transmissions in a TDM fashion. This joint algorithm takes advantage of the multi-user content diversity, and maximizes the network total utility (i.e., minimize the network total distortion), while satisfying the delivery deadline constraints. Simulations showed that the proposed algorithm significantly outperforms the constant rate provision algorithm.

Coping with handover effects in video streaming over cellular networks

The 3rd generation partnership project (3GPP) has defined the protocols and codecs for implementing media streaming services over packet-switched 3G mobile networks. The specification is based on IETF RFCs on audio/video transport.It also adds new features to achieve better adaptation to the mobile network environment. In this paper, we propose an algorithm for handover detection and fast buffer refill that is based on the existing feedback and signaling mechanisms. The proposed algorithm refills the receiver buffer at a faster pace during a limited time frame after a hard handover is detected in order to achieve higher video quality.