Hierarchical Scene Analysis Method for Audio Sensor Networks

Abstract： A hierarchical method for scene analysis in audio sensor networks is proposed. This meth-od consists of two stages： element detection stage and audio scene analysis stage. In the former stage, the basic audio elements are modeled by the HMM models and trained by enough samples off-line, and we adaptively add or remove basic ele- ment from the targeted element pool according to the time, place and other environment parameters. In the latter stage, a data fusion algorithm is used to combine the sensory information of the same ar-ea, and then, a role-based method is employed to analyze the audio scene based on the fused data. We conduct some experiments to evaluate the per-formance of the proposed method that about 70% audio scenes can be detected correctly by this method. The experiment evaluations demonstrate that our method can achieve satisfactory results.

Study on an Audio and Video Network Monitoring System for Weather Modification Operation

An audio and video network monitoring system for weather modification operation transmitting information by 3G, ADSL and Internet has been developed and applied in weather modification operation of Tai＇an City. The all-in-one machine of 3G audio and video network highly integrates all front-end devices used for audio and video collection, communication, power supply and information storage, and has advantages of wireless video transmission, clear two-way voice intercom with the command center, waterproof and dustproof function, simple operation, good portability, and long working hours. Compression code of the system is transmitted by dynamic bandwidth, and compression rate varies from 32 kbps to 4 Mbps under different network conditions. This system has forwarding mode, that is, monitoring information from each front-end monitoring point is trans- mitted to the server of the command center by 3G/ADSL, and the server codes＇and decodes again, then beck-end users call images from the serv- er, which can address 3G network stoppage caused by many users calling front-end video at the same time. In addition, the system has been ap- plied in surface weather modification operation of Tai＇an City, and has made a great contribution to transmitting operation orders in real time, monitoring, standardizing and recording operating process, and improving operating safety.

Nonlinear Prediction with Deep Recurrent Neural Networks for Non-Blind Audio Bandwidth Extension

Non-blind audio bandwidth extension is a standard technique within contemporary audio codecs to efficiently code audio signals at low bitrates. In existing methods, in most cases high frequencies signal is usually generated by a duplication of the corresponding low frequencies and some parameters of high frequencies. However, the perception quality of coding will significantly degrade if the correlation between high frequencies and low frequencies becomes weak. In this paper, we quantitatively analyse the correlation via computing mutual information value. The analysis results show the correlation also exists in low frequency signal of the context dependent frames besides the current frame. In order to improve the perception quality of coding, we propose a novel method of high frequency coarse spectrum generation to improve the conventional replication method. In the proposed method, the coarse high frequency spectrums are generated by a nonlinear mapping model using deep recurrent neural network. The experiments confirm that the proposed method shows better performance than the reference methods.

Audio Vivid标准关键技术研究及系统试验

本文在对三维声行业标准《三维声编解码及渲染》(Audio Vivid)深入研究的基础上,分析三维声编解码和渲染端到端技术框架,介绍基于神经网络的通用码率音频编码、元数据编码、扬声器渲染和双耳渲染等关键技术,给出卡塔尔世界杯期间Audio Vivid标准端到端技术试验情况,为Audio Vivid标准应用部署提供技术参考。

Environmental Sound Event Detection in Wireless Acoustic Sensor Networks for Home Telemonitoring

In this paper, we present an approach to improve the accuracy of environmental sound event detection in a wireless acoustic sensor network for home monitoring. Wireless acoustic sensor nodes can capture sounds in the home and simultaneously deliver them to a sink node for sound event detection. The proposed approach is mainly composed of three modules, including signal estimation, reliable sensor channel selection, and sound event detection. During signal estimation, lost packets are recovered to improve the signal quality. Next, reliable channels are selected using a multi-channel cross-correlation coefficient to improve the computational efficiency for distant sound event detection without sacrificing performance. Finally, the signals of the selected two channels are used for environmental sound event detection based on bidirectional gated recurrent neural networks using two-channel audio features. Experiments show that the proposed approach achieves superior performances compared to the baseline.

Multimedia Streaming for Ad Hoc Wireless Mesh Networks Using Network Coding

Over the past years, we have witnessed an explosive growth in the use of multimedia applications such as audio and video streaming with mobile and static devices. Multimedia streaming applications need new approaches to multimedia transmissions to meet the growing volume demand and quality expectations of multimedia traffic. This paper studies network coding which is a promising paradigm that has the potential to improve the performance of networks for multimedia streaming applications in terms of packet delivery ratio (PDR), latency and jitter. This paper examines several network coding protocols for ad hoc wireless mesh networks and compares their performance on multimedia streaming applications with optimized broadcast protocols, e.g., BCast, Simplified Multicast Forwarding (SMF), and Partial Dominant Pruning (PDP). The results show that the performance increases significantly with the Random Linear Network Coding (RLNC) scheme.

A Novel Two-Layer Model for Overall Quality Assessment of Multichannel Audio

With the development of multichannel audio systems, corresponding audio quality assessment techniques, especially the objective prediction models, have received increasing attention. Existing methods, such as PEAQ(Perceptual Evaluation of Audio Quality) recommended by ITU, usually lead to poor results when assessing multichannel audio, which have little correlation with subjective scores. In this paper, a novel two-layer model based on Multiple Linear Regression(MLR) and Neural Network(NN) is proposed. Through the first layer, two indicators of multichannel audio, Audio Quality Score(AQS) and Spatial Perception Score(SPS) are derived, and through the second layer the overall score is output. The final results show that this model can not only improve the correlation with the subjective test score by 30.7% and decrease the Root Mean Square Error(RMSE) by 44.6%, but also add two new indicators: AQS and SPS, which can help reflect the multichannel audio quality more clearly.

Autonomous Surveillance of Infants’ Needs Using CNN Model for Audio Cry Classification

Infants portray suggestive unique cries while sick, having belly pain, discomfort, tiredness, attention and desire for a change of diapers among other needs. There exists limited knowledge in accessing the infants’ needs as they only relay information through suggestive cries. Many teenagers tend to give birth at an early age, thereby exposing them to be the key monitors of their own babies. They tend not to have sufficient skills in monitoring the infant’s dire needs, more so during the early stages of infant development. Artificial intelligence has shown promising efficient predictive analytics from supervised, and unsupervised to reinforcement learning models. This study, therefore, seeks to develop an android app that could be used to discriminate the infant audio cries by leveraging the strength of convolution neural networks as a classifier model. Audio analytics from many kinds of literature is an untapped area by researchers as it’s attributed to messy and huge data generation. This study, therefore, strongly leverages convolution neural networks, a deep learning model that is capable of handling more than one-dimensional datasets. To achieve this, the audio data in form of a wave was converted to images through Mel spectrum frequencies which were classified using the computer vision CNN model. The Librosa library was used to convert the audio to Mel spectrum which was then presented as pixels serving as the input for classifying the audio classes such as sick, burping, tired, and hungry. The study goal was to incorporate the model as an android tool that can be utilized at the domestic level and hospital facilities for surveillance of the infant’s health and social needs status all time round.

网络多媒体传输协议在物联网音频监控系统中的应用

介绍网络多媒体传输协议的基本原理,探讨其在音频监控中的应用,包括音频数据捕获、编码、传输、接收及解码等环节。基于此,设计一个基于网络多媒体传输协议的物联网音频监控系统架构,并对其功能进行设计。最后进行系统运行测试,验证该架构的可行性和有效性。

AI技术在广播电视和网络视听领域的应用研究

以人工智能(Artificial Intelligence,AI)技术在广播电视和网络视听领域的应用为重点,首先全面介绍人工智能的概念、演进方向、产业发展现状,其次重点分析AI技术在广播电视和网络视听领域的应用情况,包括智能内容审核、智能效果评估、智能推荐、智能剪辑等技术与系统在广播电视和网络视听领域的具体应用,推动广播电视和网络视听行业向智能化、自动化、智慧化方向发展。

融合注意力与多分支膨胀卷积的音频隐写算法

为提升音频隐写算法的透明性与安全性,提出了一种将多分支膨胀卷积网络(Multi-Branch Dilated Convolutional Network,MBDC)与残差瓶颈注意力模块相结合的高透明性、高鲁棒性和高隐藏容量的音频隐写算法。编码器采用不同膨胀率组成的多分支膨胀卷积网络进行局部编码,完成对音频的嵌入,在音频采样率相同时,可用更少的参数获得更大的感受野,更全面地捕捉音频信号的上下文信息。在编码器与解码器后增加残差注意力模块,增加了网络对音频关键特征的辨别能力,提高了音频隐写算法的透明性与隐藏容量。将算法在多个音频数据集中进行实验,结果表明,该隐写算法具有较好的泛化能力,与传统隐写算法和其他神经网络模型相比,具有更好的透明性与隐藏容量,同时该算法对不同噪声干扰具有良好的鲁棒性。

基于“声”态环境下的网络音频平台播音创作优化方法研究

随着互联网的不断发展和相关技术水平的提升,目前人们接收信息的方式和习惯发生了重大改变。以移动设备和网络的发展为例,快捷的网络服务和高质量设备的使用导致人们的收听状态呈现移动化、碎片化特征,而网络音频平台作为播音节目的主要载体,在内容的生产创作上需要进一步满足时代发展和人们收听的需求。本文从“声”态环境出发,针对网络音频平台播音创作的发展需求,提出了具体的优化策略,为“声”态环境下的播音工作者提供新的创作思路,促进行业的未来发展。

天津市地震局应急视频会议系统组网研究

为全面提升地震应急指挥能力和协同应急通讯效率,天津市地震局应急指挥大厅基于跨网段多层级视频融合技术完善天津市地震应急视频指挥调度网络,构建“平战结合”双模式。通过探究跨网段通讯、多层级音视频信号转发、多点控制单元(MCU)级联等技术,实现天津局与中国局、市政府、市应急管理局以及辖区内各应急视频节点互连互通,为天津市地震应急快速响应提供高效的通信保障。

基于欠定盲源分离和深度学习的生猪状态音频识别

【目的】为解决群养环境下生猪音频难以分离与识别的问题,提出基于欠定盲源分离与E C A-EfficientNetV2的生猪状态音频识别方法。【方法】以仿真群养环境下4类生猪音频信号作为观测信号,将信号稀疏表示后,通过层次聚类估计出信号混合矩阵,并利用lp范数重构算法求解lp范数最小值以完成生猪音频信号重构。将重构信号转化为声谱图,分为进食声、咆哮声、哼叫声和发情声4类,利用ECA-EfficientNetV2网络模型识别音频,获取生猪状态。【结果】混合矩阵估计的归一化均方误差最低为3.266×10^(−4),分离重构的音频信噪比在3.254~4.267 dB之间。声谱图经ECA-EfficientNetV2识别检测,准确率高达98.35%;与经典卷积神经网络ResNet50和VGG16对比,准确率分别提升2.88和1.81个百分点;与原EfficientNetV2相比,准确率降低0.52个百分点,但模型参数量减少33.56%,浮点运算量(FLOPs)降低1.86 G,推理时间减少9.40 ms。【结论】基于盲源分离及改进EfficientNetV2的方法,轻量且高效地实现了分离与识别群养生猪音频信号。

基于音频处理的广电网络通信接入技术研究

围绕音频处理技术在广电网络通信接入中的应用展开研究,重点探讨音频编码、降噪增强、网络传输优化等关键技术,并结合案例分析具体的应用方案。引入Opus编码、RNNoise降噪、网页实时通信(Web Real-Time Communication,WebRTC)自适应抖动缓冲及丢包隐藏等先进技术后,方案在音质、稳定性、实时性等方面取得了显著提升。

数字音频传输系统的时钟同步

归纳了数字音频传输系统中常用的时钟信号,针对常见的时钟同步问题提出相应解决方式及措施,主要探讨了同步时钟系统的主时钟选择、异步时钟系统间的信号互通及基于云平台的音视频同步技术,以及解决时钟抖动引起的量化误差的方法。

人工智能技术在广播电视和网络视听中的应用研究

介绍人工智能(Artificial Intelligence,AI)的概念、技术发展现状及产业发展现状,重点分析智能推荐、场景生成、人脸合成、视频修复及虚拟数字人技术在广播电视和网络视听中的具体应用,助力营造全社会AI视听业务发展生态。

虚拟数字人技术在广播电视领域的应用

介绍虚拟数字人的概念、技术发展历程、主要类型以及产业发展现状,重点分析虚拟数字人技术在新闻、体育、综艺及短视频等领域的具体应用,为虚拟数字人与广播电视及网络视听融合发展提供参考。

智能语音技术在广播电视和网络视听领域的应用研究

以智能语音技术在广播电视和网络视听领域的应用为研究重点,探讨智能语音技术的概念、发展现状及应用领域,重点分析智能语音技术在广播电视和网络视听领域的应用,并举实例进行说明,旨在为智能语音技术在广播电视和网络视听领域的研究和应用提供借鉴。

面向网络多媒体的音频隐写术应用

音频隐写术利用音频信号的冗余性和人类听觉系统的特性,将秘密信息嵌入音频中进行隐蔽传输。根据不同的嵌入方式,音频隐写术可分为时域隐写术、频域隐写术和扩频隐写术等。不同的音频隐写术各具特点,并适用于不同场景。在网络多媒体环境中,音频隐写术在版权保护、身份验证及军事通信等领域发挥重要作用,但也面临着嵌入容量与健壮性、实时性与安全性之间的平衡等挑战。基于此,文章深入探讨音频隐写术的原理、特点、分类以及在网络多媒体环境中的应用,展望其在未来信息安全领域的发展前景。