Real-time Audio &Video Transmission System Based on Visible Light Communication

With the increasing popularity of solid sate lighting devices, Visible Light Communication (VLC) is globally recognized as an advanced and promising technology to realize short-range, high speed as well as large capacity wireless data transmission. In this paper, we propose a prototype of real-time audio and video broadcast system using inexpensive commercially available light emitting diode (LED) lamps. Experimental results show that real-time high quality audio and video with the maximum distance of 3 m can be achieved through proper layout of LED sources and improvement of concentration effects. Lighting model within room environment is designed and simulated which indicates close relationship between layout of light sources and distribution of illuminance.

Web-based interactive visualization of 3D video mosaics using X3D standard

We present a method of 3D image mosaicing for real 3D representation of roadside buildings, and implement a Web-based interactive visualization environment for the 3D video mosaics created by 3D image mosaicing. The 3D image mo- saicing technique developed in our previous work is a very powerful method for creating textured 3D-GIS data without excessive data processing like the laser or stereo system. For the Web-based open access to the 3D video mosaics, we build an interactive visualization environment using X3D, the emerging standard of Web 3D. We conduct the data preprocessing for 3D video mosaics and the X3D modeling for textured 3D data. The data preprocessing includes the conversion of each frame of 3D video mosaics into concatenated image files that can be hyperlinked on the Web. The X3D modeling handles the representation of concatenated images using necessary X3D nodes. By employing X3D as the data format for 3D image mosaics, the real 3D representation of roadside buildings is extended to the Web and mobile service systems.

Adaptive Learning Video Streaming with QoE in Multi-Home Heterogeneous Networks

In recent years,real-time video streaming has grown in popularity.The growing popularity of the Internet of Things(IoT)and other wireless heterogeneous networks mandates that network resources be carefully apportioned among versatile users in order to achieve the best Quality of Experience(QoE)and performance objectives.Most researchers focused on Forward Error Correction(FEC)techniques when attempting to strike a balance between QoE and performance.However,as network capacity increases,the performance degrades,impacting the live visual experience.Recently,Deep Learning(DL)algorithms have been successfully integrated with FEC to stream videos across multiple heterogeneous networks.But these algorithms need to be changed to make the experience better without sacrificing packet loss and delay time.To address the previous challenge,this paper proposes a novel intelligent algorithm that streams video in multi-home heterogeneous networks based on network-centric characteristics.The proposed framework contains modules such as Intelligent Content Extraction Module(ICEM),Channel Status Monitor(CSM),and Adaptive FEC(AFEC).This framework adopts the Cognitive Learning-based Scheduling(CLS)Module,which works on the deep Reinforced Gated Recurrent Networks(RGRN)principle and embeds them along with the FEC to achieve better performances.The complete framework was developed using the Objective Modular Network Testbed in C++(OMNET++),Internet networking(INET),and Python 3.10,with Keras as the front end and Tensorflow 2.10 as the back end.With extensive experimentation,the proposed model outperforms the other existing intelligentmodels in terms of improving the QoE,minimizing the End-to-End Delay(EED),and maintaining the highest accuracy(98%)and a lower Root Mean Square Error(RMSE)value of 0.001.

非匀速条件下车辆底盘超近距成像测量方法

针对车辆非匀速条件下轴距视觉测量及轴型识别所面临的超近距、大幅面、低畸变成像及视觉测量问题,提出一种适用于非匀速条件下的车辆底盘结构超近距成像测量方法。利用鱼眼相机采集运动车辆底盘的超广角图像序列,通过一种改进的多采样点圆拟合方法实现图像感兴趣区域的提取。建立三维等距成像模型对鱼眼图像进行畸变校正。针对车辆非匀速行驶带来的视频帧重叠度差异,采用基于图像内容的重叠度分析技术,实现了多步分层的关键帧提取,有效避免了车速不稳定对车辆底盘结构测量的影响。现场实验结果表明:该方法可在非匀速条件下,以20~50 cm的成像距离获得清晰、低畸变的完整车辆底盘图像,车辆轴距测量误差均值约为5%。

RTP payload format for H.264/SVC scalable video coding

The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team’s work. In its present working draft, the higher level syntax of SVC follows the design principles of H.264/AVC. Self-contained network abstraction layer units (NAL units) form natural entities for packetization. The SVC specification is by no means finalized yet, but nevertheless the work towards an optimized RTP payload format has already started. RFC 3984, the RTP payload specification for H.264/AVC has been taken as a starting point, but it became quickly clear that the scalable features of SVC require adaptation in at least the areas of capability/operation point signaling and documentation of the extended NAL unit header. This paper first gives an overview of the history of scalable video coding, and then reviews the video coding layer (VCL) and NAL of the latest SVC draft specification. Finally, it discusses different aspects of the draft SVC RTP payload format, in- cluding the design criteria, use cases, signaling and payload structure.

Real-time mosaicking for infrared videos from an oblique sweeping camera

Image mosaicking is widely used in Geographic Information Systems(GISs)for largescale ground surface analysis.However,most existing mosaicking methods can only be used in offline processing due to the enormous amounts of computation.In this paper,we propose a novel and practical algorithm for real-time infrared video mosaicking.To achieve this,a novel fast template matching algorithm based on Sum of Cosine Differences(SCD)is proposed to coarsely match the sequential images.The high speed of the proposed template matching algorithm is obtained by computing correlation with Fast Fourier Transform(FFT).We also propose a novel fast Least Squares Matching(LSM)algorithm for inter-frame fine registration,which can significantly reduce the computation without degrading the matching accuracy.In addition,the proposed fast LSM can effectively adapt for noise degradation and geometric distortion.Based on the proposed fast template matching algorithm and fine registration algorithm,we develop a practical real-time mosaicking approach which can produce seamless mosaic image highly efficiently.Experiments on synthetic and real-world datasets demonstrate that the proposed algorithm is not just computationally efficient but also robust against various noise distortions.

Mosaic视频导航系统的分析

本文主要介绍了基于DVB标准EN 300 468的Mosaic视频导航系统。Mosaic是一种利用多个视频图像组合进行节目导航的机制,可以把它看成是一种高级的电子节目指南。文章先介绍了Mosaic节目视频导航机制的原理,然后分析了Mosaic描述符的数据结构,并补充必要的私有描述符,最后给出实现方法概要。

View Direction Adaptive 360 Degree Video Streaming System Based on Projected Area

360 video streaming services over the network are becoming popular. In particular, it is easy to experience 360 video through the already popular smartphone. However, due to the nature of 360 video, it is difficult to provide stable streaming service in general network environment because the size of data to send is larger than that of conventional video. Also, the real user's viewing area is very small compared to the sending amount. In this paper, we propose a system that can provide high quality 360 video streaming services to the users more efficiently in the cloud. In particular, we propose a streaming system focused on using a head mount display (HMD).

A Full View Video Navigation System

This paper presents a full view video navigation platform and the approach to buildup such a platform.Unlike the traditional full view navigation system,which always bases on image panorama,our system is based on full view video created by video mosaics.Neither the current concept of video mosaics,which means posting together video frames to get a wide view,while these frames are from the same video.Video mosaics in this paper mean stitching different videos together to get a full view video.Videos,together recording 360° of the scenery,are captured through a certain route by digital cameras arranged in a circle around the view point.Then break up these videos into frame sequence.Frame coming from different videos with the same timestamp are stitched into a panoramic image.Consider these panoramic images as frames;montage these panoramic frames into a full view video,by sequence of time.While displaying the full view video people could decide to go forward,or stop to see around,or proceed in a certain direction.While talking about full view navigation,we are mostly talking about image-based cylindrical or spherical panorama [2].One way to create such a panorama is,after a series of translation,stitching together a group of images taken by panning camera [1].Each of these images presents part of the scene and with certain degrees of overlap to adjacent ones.After stitching,we get a panorama image with width is much bigger than the raw images,map it onto a cylinder or sphere surface is a better way to show it.The mapped one is called cylindrical panorama or spherical panorama.While navigating,taking sphere panorama as an example,people were supposed to standing at the center of the sphere,by rotating the sphere,different view coming into eyes.This kind of navigation makes people feel as the same as standing in the real world.But one limitation is that the step is stuck to one point,could see around but not move forward or back(there is moving forward and back by zooming in and zooming out respectively,but it's not truly move).In our work,videos are used as the stitching source instead of images.Video,or could say a sequence of images taken at continuous view point(actually it is the definition of 'video',and is the saying we are going to take advantage of in this paper),are captured by a camera circle,and are stitched into a full view video.Frames coming from different video with the same timestamp are stitched into a panorama frame.Transform the processed panorama frames into a video.

基于水下机器人的海底视频图像拼接技术探讨

运用水下机器人(ROV、AUV)配备的高清数码相机拍摄深海底的视频图像(视频、照片)有其自身的特点,如获取难度较高、覆盖面积小、光照不均匀、特征点少、数据量大等。为了获取整体大范围的海底影像,使海洋科研人员更直观、全面地认识海底微地形地貌,对其底质状况进行分析,以更详细地了解海洋底栖生物的生态环境,需要对这些拍摄的海底视频图像进行拼接处理。本文提出一种基于水下机器人的视频图像拼接流程和处理方法,首先对水下机器人拍摄的海底视频数据进行关键帧提取,得到带时间戳的序列图像;然后导入姿态数据中的偏航角对序列图像进行方位角恢复;最后基于定位数据批量地对经过方位角处理后的序列图像进行拼接和融合匀色,最终获得拍摄范围内的海底全景拼接图像。基于本文提出的图像拼接方法,对某科考航次中获取的视频图像进行拼接处理实验,取得了较好的拼接效果,验证了本文提出的方法的可行性。

Performance Analysis of Hybrid RR Algorithm for Anomaly Detection in Streaming Data

Automated live video stream analytics has been extensively researched in recent times.Most of the traditional methods for video anomaly detection is supervised and use a single classifier to identify an anomaly in a frame.We propose a 3-stage ensemble-based unsupervised deep reinforcement algorithm with an underlying Long Short Term Memory(LSTM)based Recurrent Neural Network(RNN).In the first stage,an ensemble of LSTM-RNNs are deployed to generate the anomaly score.The second stage uses the least square method for optimal anomaly score generation.The third stage adopts award-based reinforcement learning to update the model.The proposed Hybrid Ensemble RR Model was tested on standard pedestrian datasets UCSDPed1,USDPed2.The data set has 70 videos in UCSD Ped1 and 28 videos in UCSD Ped2 with a total of 18560 frames.Since a real-time stream has strict memory constraints and storage issues,a simple computing machine does not suffice in performing analytics with stream data.Hence the proposed research is designed to work on a GPU(Graphics Processing Unit),TPU(Tensor Processing Unit)supported framework.As shown in the experimental results section,recorded observations on framelevel EER(Equal Error Rate)and AUC(Area Under Curve)showed a 9%reduction in EER in UCSD Ped1,a 13%reduction in ERR in UCSD Ped2 and a 4%improvement in accuracy in both datasets.

基于CUDA的全景视频实时监控系统设计

针对传统监控方案人机交互感受较差、存在视觉盲区等缺陷,以及传统全景视频拼接技术难以实现实时性输出等问题,设计了一种基于CUDA编译模型的全景视频实时监控系统。通过初始化阶段计算出每一路相机获取的图像的像素点位置变换关系并保存在映射表中,在重叠区域寻找一条最佳接缝线并生成接缝线掩码图,利用CUDA编译模型实现全景图像的实时处理。实验结果表明,该监控系统具有良好的视觉性,并达到了全景视频实时性输出的工程要求。

Augmented virtual environment: fusion of real-time video and 3D models in the digital earth system

An Augmented virtual environment(AVE)is concerned with the fusion of real-time video with 3D models or scenes so as to augment the virtual environment.In this paper,a new approach to establish an AVE with a wide field of view is proposed,including real-time video projection,multiple video texture fusion and 3D visualization of moving objects.A new diagonally weighted algorithm is proposed to smooth the apparent gaps within the overlapping area between the two adjacent videos.A visualization method for the location and trajectory of a moving virtual object is proposed to display the moving object and its trajectory in the 3D virtual environment.The experimental results showed that the proposed set of algorithms are able to fuse multiple real-time videos with 3D models efficiently,and the experiment runs a 3D scene containing two million triangles and six real-time videos at around 55 frames per second on a laptop with 1GB of graphics card memory.In addition,a realistic AVE with a wide field of view was created based on the Digital Earth Science Platform by fusing three videos with a complex indoor virtual scene,visualizing a moving object and drawing its trajectory in the real time.

Supporting multi-user real-time VBR video service using dynamic bandwidth management based on prediction

Real-time variable bit rate（VBR） video is expected to take a significant portion of multimedia applications.However,plentiful challenges to VBR video service provision have been raised for its characteristic of high traffic abruptness.To support multi-user real-time VBR video transmission with high bandwidth utilization and satisfied quality of service（QoS）,this article proposes a practical dynamic bandwidth management scheme.This scheme forecasts future media rate of VBR video by employing time-domain adaptive linear predictor and using media delivery index（MDI） as both QoS measurement and complementary management reference.In addition,to support multi-user application,an adjustment priorities classified strategy is also put forward.Finally,a test-bed based on this management scheme is established.The experimental results demonstrate that the scheme proposed in this article is efficient with bandwidth utilization increased by 20%-60% compared to a fixed service rate and QoS guaranteed.

Real-time ultra-wideband video streaming in long-reach passive optical networks with wireless radiation in the 10 and 60 GHz bands

Real-time video streaming using ultra-wideband(UWB) technology is experimentally demonstrated along long-reach passive optical networks(LR-PONs) with different wired and wireless reaches. Experimental tests using external and direct modulation with UWB wireless radiation in the 10- and 60-GHz bands are performed. An ultra-bendable fiber is also considered for a last-mile distribution. The video quality at the output of the optical fiber infrastructure of the LR-PON is assessed using the error vector magnitude(EVM), and the link quality indicator(LQI) is used as a figure of merit after wireless radiation. An EVM below –17 dB is achieved for both externally and directly modulated LR-PONs comprising up to 125 km of optical fiber. EVM improvement is observed for longer LR-PONs when directly modulated lasers(DMLs) are used because of the amplitude gain provided by the combined effect of dispersion and DML's chirp. Compared with optical back-to-back operation, the LQI level degrades to the maximum around 20% for LR-PONs ranging between 75 and 125 km of fiber reach and with a wireless coverage of 2 m in the 10-GHz UWB band. The same level of LQI degradation is observed using the 60-GHz UWB band with a LR-PON integrating 101 km of access network, a last-mile distribution using ultra-bendable fiber, and a 5.2-m wireless link.

基于Zynq-7000的宽幅视频处理系统设计与实现

针对安防监控与智慧交通领域无盲区视频摄录的需求,基于Zynq-7000平台优势进行宽幅视频处理软硬件系统的设计。通过研究基于嵌入式系统的多路视频实时拼接技术,设计并实现了以XC7Z020芯片为核心的视频处理板卡,在双核ARM Cotex-A9硬核处理器端服务程序协同调度下,基于片内FPGA硬件加速能力实现多通道视频实时采集与预处理、投影变换、图像融合,进而形成完整宽幅视频画面,通过远程客户端实现定制化的视频摄录功能。测试结果表明,该系统是兼具稳定的性能和灵活区域部署能力的便携式解决方案。

一种改进的摄像头视频实时拼接方法

针对普通摄像头水平视角较小的问题,通过同时采集具有一定重叠区域的摄像头视频帧图像,基于尺度不变特征变换(Scale Invariant Feature Transform,SIFT)特征点,用二分哈希搜索算法(Dichotomy Based Hash,DBH)进行匹配,用随机采样一致(Random Sample Consensus,RANSAC)算法消除误匹配,得到帧图像拼接映射关系。实验结果表明,该方法能有效地实现摄像头视频实时拼接,克服了既有方法在重叠区域小于20%时失效的不足,在重叠区域为10%左右时仍能取得有效的拼接。

采用视频拼图方法构建高分辨率全景视频监控系统

与普通视频监控系统只能实现单向监控不同,全景视频监控系统可以实现360°全向监控。设计并实现了一种嵌入式高分辨率全景视频监控系统KD-PVS。重点介绍了KD-PVS中多个摄像头的空间位置设计、视频图像变换与拼接算法。KD-PVS通过对多个摄像头采集的视频进行实时变换与拼接以生成全景视频。该系统可方便应用于金融系统、仓库、监狱和移动监控等多种场合,尤其适用于室内监控。

实时视场拼接系统的设计与实现

当要求的场景尺寸超出一个光学传感器的范围时,同时取得完整的场景就成为一个难点。针对这一问题,采用多个光学传感器同时对场景进行采集,可以得到几幅互相有一定重叠的场景图像。应用改进的相位相关算法对重叠图像进行快速配准,应用渐入渐出融合算法消除拼缝,实现无缝大视场拼接。并将整套算法在以TMS320DM642为核心处理器的平台上实现,得以构成一个小型化视频拼接系统。实验结果表明,该系统可以自动地对存在一定重叠和旋转的两幅768×494分辨率、25帧/秒的视频图像进行拼接,获得无缝、清晰的大视场视频图像,满足系统实时性的要求。

高实时性F-SIFT图像拼接算法

大场景视频拼接技术被广泛地应用于不同的领域,其应用要求精度高,实时性好。在对频域相关法、基于SIFT的图像拼接算法等方法进行深入研究的基础上,对其在配准精度、速度等方面进行改进与融合,提出一种具有高拼接性能、适应实时性拼接需求的F-SIFT图像拼接算法。该算法结合基于区域的图像配准方法和基于特征的SIFT两者优势,采用基于子图像块的频域相位相关算法,兼顾FFT的快速运算特点,提高图像拼接算法运算速度。实验对算法进行测试,验证F-SIFT算法的有效性。