Geographical Information Visualization on a Panoramic Sphere in an Immersive Environment

In this paper,we propose a novel approach to visualizing global geographical information:a panoramic sphere in an immersive environment.The whole geographical surface can be observed through the rotating of heads as the viewpoint of the panoramic sphere is inside the sphere.We compared three approaches to visualizing the earth for rendering the geographical information in a virtual reality environment.On the tasks of terrestrial and marine geographical information,we compare the visualization effects on a)a globe,b)a flat map and c)a panoramic sphere.Terrestrial geographical information tasks include the area comparison and direction determination.Marine geographical information tasks contain the visualization of sea surface temperature and sea surface currents.For terrestrial geographical information tasks,the experimental results show that the panoramic sphere outperforms the globe and the flat map,with a higher average accuracy and a shorter time.On marine geographical information task,the panoramic sphere visualization is also superior to the flat map and the globe in an immersive environment for the sea surface temperature data and the sea surface current fields.In all three visualization experiments,the panoramic sphere is most preferred by the participants,particularly for global,transcontinental and transoceanic needs.

On‐device audio‐visual multi‐person wake word spotting

Audio‐visual wake word spotting is a challenging multi‐modal task that exploits visual information of lip motion patterns to supplement acoustic speech to improve overall detection performance.However,most audio‐visual wake word spotting models are only suitable for simple single‐speaker scenarios and require high computational complexity.Further development is hindered by complex multi‐person scenarios and computational limitations in mobile environments.In this paper,a novel audio‐visual model is proposed for on‐device multi‐person wake word spotting.Firstly,an attention‐based audio‐visual voice activity detection module is presented,which generates an attention score matrix of audio and visual representations to derive active speaker representation.Secondly,the knowledge distillation method is introduced to transfer knowledge from the large model to the on‐device model to control the size of our model.Moreover,a new audio‐visual dataset,PKU‐KWS,is collected for sentence‐level multi‐person wake word spotting.Experimental results on the PKU‐KWS dataset show that this approach outperforms the previous state‐of‐the‐art methods.

Securing Technique Using Pattern-Based LSB Audio Steganography and Intensity-Based Visual Cryptography

With the increasing need of sensitive or secret data transmission through public network,security demands using cryptography and steganography are becoming a thirsty research area of last few years.These two techniques can be merged and provide better security which is nowadays extremely required.The proposed system provides a novel method of information security using the techniques of audio steganography combined with visual cryptography.In this system,we take a secret image and divide it into several subparts to make more than one incomprehensible sub-images using the method of visual cryptography.Each of the sub-images is then hidden within individual cover audio files using audio steganographic techniques.The cover audios are then sent to the required destinations where reverse steganography schemes are applied to them to get the incomprehensible component images back.At last,all the sub-images are superimposed to get the actual secret image.This method is very secure as it uses a two-step security mechanism to maintain secrecy.The possibility of interception is less in this technique because one must have each piece of correct sub-image to regenerate the actual secret image.Without superimposing every one of the sub-images meaningful secret images cannot be formed.Audio files are composed of densely packed bits.The high density of data in audio makes it hard for a listener to detect the manipulation due to the proposed time-domain audio steganographic method.

Integrating Audio-Visual Features and Text Information for Story Segmentation of News Video

Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The proposed approach detects the topic-caption frames, and integrates them with silence clips detection results, as well as shot segmentation results to locate the news story boundaries. The integration of audio-visual features and text information overcomes the weakness of the approach using only image analysis techniques. On test data with 135 400 frames, when the boundaries between news stories are detected, the accuracy rate 85.8% and the recall rate 97.5% are obtained. The experimental results show the approach is valid and robust.

Audio-visual emotion recognition with multilayer boosted HMM

Emotion recognition has become an important task of modern human-computer interac- tion. A multilayer boosted HMM （ MBHMM ） classifier for automatic audio-visual emotion recognition is presented in this paper. A modified Baum-Welch algorithm is proposed for component HMM learn- ing and adaptive boosting （AdaBoost） is used to train ensemble classifiers for different layers （cues）. Except for the first layer, the initial weights of training samples in current layer are decided by recognition results of the ensemble classifier in the upper layer. Thus the training procedure using current cue can focus more on the difficult samples according to the previous cue. Our MBHMM clas- sifier is combined by these ensemble classifiers and takes advantage of the complementary informa- tion from multiple cues and modalities. Experimental results on audio-visual emotion data collected in Wizard of Oz scenarios and labeled under two types of emotion category sets demonstrate that our approach is effective and promising.

The Effect of Audio Visual Entrainment on Pre-Attentive Dysfunctional Processing to Stressful Events in Anxious Individuals

Experimental single case studies on automatic processing of emotion were carried on a sample of people with an anxiety disorder. Participants were required to take three Audio Visual Entrainment (AVE) sessions to test for anxiety reduction as proclaimed by some academic research. Explicit reports were measured as well as pre-attentive bias to stressing information by using affective priming studies before and after AVE intervention. Group analysis shows that indeed AVEs program applications do reduce anxiety producing significant changes over explicit reports on anxiety levels and automatic processing bias of emotion. However, case by case analysis of six anxious participants shows that even when all of the participants report emotional improvement after intervention, not all of them reduce or eliminate dysfunctional bias to stressing information. Rather, they show a variety of processing styles due to intervention and some of them show no change at all. Implications of this differential effect to clinical sets are discussed.

Prioritized MPEG-4 Audio-Visual Objects Streaming over the DiffServ

The object-based scalable coding in MPEG-4 is investigated, and a prioritized transmission scheme of MPEG-4 audio-visual objects （AVOs） over the DiffServ network with the QoS guarantee is proposed. MPEG-4 AVOs are extracted and classified into different groups according to their priority values and scalable layers （visual importance）. These priority values are mapped to the 1P DiffServ per hop behaviors （PHB）. This scheme can selectively discard packets with low importance, in order to avoid the network congestion. Simulation results show that the quality of received video can gracefully adapt to network state, as compared with the ‘best-effort＇ manner. Also, by allowing the content provider to define prioritization of each audio-visual object, the adaptive transmission of object-based scalable video can be customized based on the content.

Immersive visualization of 3D subsurface ground model developed from sparse boreholes using virtual reality(VR)

Analytics and visualization of multi-dimensional and complex geo-data,such as three-dimensional(3D)subsurface ground models,is critical for development of underground space and design and construction of underground structures(e.g.,tunnels,dams,and slopes)in engineering practices.Although complicated 3D subsurface ground models now can be developed from site investigation data(e.g.,boreholes)which is often sparse in practice,it remains a great challenge to visualize a 3D subsurface ground model with sophisticated stratigraphic variations by conventional two-dimensional(2D)geological cross-sections.Virtual reality(VR)technology,which has an attractive capability of constructing a virtual environment that links to the physical world,has been rapidly developed and applied to visualization in various disciplines recently.Leveraging on the rapid development of VR,this study proposes a framework for immersive visualization of 3D subsurface ground models in geo-applications using VR technology.The 3D subsurface model is first developed from limited borehole data in a data-driven manner.Then,a VR system is developed using related software and hardware devices currently available in the markets for immersive visualization and interaction with the developed 3D subsurface ground model.The results demonstrate that VR visualization of the 3D subsurface ground model in an immersive environment has great potential in revolutionizing the geo-practices from 2D cross-sections to a 3D immersive virtual environment in digital era,particularly for the emerging digital twins.

基于ⅢF A/V规范和Avalon系统的大学图书馆视听数据库建设研究

随着中国网络基础设施的不断改善,视听媒体在年轻一代中非常流行,给以文本资源为主的图书馆带来了挑战。本研究旨在探究国内外大学图书馆视听资源数据库建设的现状,借鉴ⅢF规范在图像资源管理方面的成功经验和各种视听保存社区的实践,提出基于ⅢF A/V规范与开源软件的中国大学图书馆视听资源管理方法。通过分析华东师范大学图书馆在视听资源保存、流媒体发布、时间轴气泡注释、转录、视听结构化和开放共享方面的实践,进行实证研究。

二重情境:数字视听文化中的身份构建与认同疏离

数字视听媒介及其文化的发展创造了新的情境,推动用户产生新的媒介实践,但也带来了新的问题和现象,即情境生成逻辑向流媒体用户让渡的转变、情境功能作为交往工具的偏移,以及情境秩序在去公共性过程中权力再结构化的取向。其中隐含着情境之于用户身份构建的二重性:一方面,流媒体用户依据情境构建身份,身份的构建过程进一步激发了情境的创造;另一方面,这些被构建的身份呈现出去情境化的流动趋向,造成了身份与认同的疏离。对此,文章在厘清数字视听文化新情境特征的基础上,阐释流媒体用户基于身份的情境互动过程,以及身份认同疏离的成因及危机。更进一步,文章尝试将新媒介—新情境—新行为的线性模式延展为更具解释力的用户主导循环模式,并提出通过共识的凝聚、公共性的重拾和共同体的重建,推动数字视听文化的良序发展。

智能计算时代深化电影科技创新和推进电影强国建设的思考与启示

人类社会正加快步入智能计算时代,现代智能科技对电影产业的影响持续深化和不断泛化,机器学习、深度学习、人工智能生成内容(AIGC)、大语言模型(LLM)、多模态模型等技术正在深刻影响和重构优化电影全产业链全价值链。伴随电影科技创新进入攻坚期和深水区,电影产业发展面临新形势新要求,我们要深化电影科技自主创新,积极推进电影行业的智能化升级,推进生成式人工智能、人工智能大模型、沉浸式视听等技术在我国电影行业的定制设计、自主研制与科学应用,有力支撑和服务社会主义电影强国建设。

基于视音互补语义清晰化的隐私视频动作识别方法

视频隐私保护是当前社会面临的重要挑战之一,对视频进行模糊处理是保护人们隐私权益的重要手段.由于模糊视频天然缺失视觉模态的信息,主流的视频动作识别算法无法取得令人满意的效果.模糊视频作为多模态介质不仅仅只有视觉模态信息,同时,也含有丰富的音频模态信息,从人类的认知角度而言,音频也是获取信息的重要来源.本文提出一种基于多模态融合的隐私视频动作识别方法,在保证不侵犯使用者隐私的前提下进行人类动作行为识别.具体来说,使用音频-视觉特征融合模块将音频模态特征图融入到视觉模态中,充分融合音视频模态的深层语义信息.除此之外,模型还引入清晰视频帧图像作为标签,在模型训练阶段监督动作识别网络的参数更新,为隐私视频动作识别网络提供清晰的语义信息.在多组隐私行为数据集上,通过大量消融和对比实验验证了所提方法的有效性.

基于扩张卷积和Transformer的视听融合语音分离方法

为了提高语音分离的效果,除了利用混合的语音信号,还可以借助视觉信号作为辅助信息。这种融合了视觉与音频信号的多模态建模方式,已被证实可以有效地提高语音分离的性能,为语音分离任务提供了新的可能性。为了更好地捕捉视觉与音频特征中的长期依赖关系,并强化网络对输入上下文信息的理解,本文提出了一种基于一维扩张卷积与Transformer的时域视听融合语音分离模型。将基于频域的传统视听融合语音分离方法应用到时域中,避免了时频变换带来的信息损失和相位重构问题。所提网络架构包含四个模块:一个视觉特征提取网络,用于从视频帧中提取唇部嵌入特征;一个音频编码器,用于将混合语音转换为特征表示;一个多模态分离网络,主要由音频子网络、视频子网络,以及Transformer网络组成,用于利用视觉和音频特征进行语音分离;以及一个音频解码器,用于将分离后的特征还原为干净的语音。本文使用LRS2数据集生成的包含两个说话者混合语音的数据集。实验结果表明,所提出的网络在尺度不变信噪比改进(Scale-Invariant Signal-to-Noise Ratio Improvement,SISNRi)与信号失真比改进(Signal-to-Distortion Ratio Improvement,SDRi)这两种指标上分别达到14.0 dB与14.3 dB,较纯音频分离模型和普适的视听融合分离模型有明显的性能提升。

视听语言智能康复技术对语言发育迟缓儿童语言功能的影响

目的 探究视听语言智能康复技术对语言发育迟缓儿童语言功能的影响,为语言发育迟缓儿童的语言功能训练提供临床参考依据。方法 选取2022年1月~2023年3月兰州大学第二医院收治的80例语言发育迟缓儿童,随机分为研究组和对照组各40例。对照组进行常规语言训练,研究组在常规语言训练基础上联合使用视听语言智能康复技术,治疗3个月。治疗前后采用Gesell发育量表评估两组的适应性能力、语言能力、个人社交能力,并采用ICF(international classification of functioning disability and health)早期语言障碍评估与干预治疗仪对儿童词语、句子理解能力进行评估。结果 治疗前两组患儿适应性能力、语言能力、个人社交能力3大能区发育商以及词语理解、句子理解能力得分无显著差异(P>0.05)。治疗后研究组适应性能力、语言能力、个人社交能力3大能区发育商显著提高(P<0.05)。两组患儿词语理解、句子理解能力得分均显著提高,且研究组显著高于对照组(P<0.05)。结论 针对语言发育迟缓儿童语言能力的改善,常规语言训练结合视听语言智能康复技术比单纯常规语言训练效果更好。

民间传说在沉浸式互动装置艺术的视觉转化与艺术表现

在当今全新的视觉文化时代,急需利用现代视觉语言为民间传说的传承与创新性转化找寻可行出路。文章阐述了沉浸式互动装置的概念以及民间传说传承传播现状,通过分析沉浸式互动装置艺术的特征,探讨出多元媒介、沉浸式体验、多媒介互动空间构建的3种融合策略,对视觉时代背景下沉浸式互动装置艺术与民间传说融合的潜在价值展开了充分的探讨,为民间传说视觉转化与传承的沉浸式互动装置艺术实现提供参考。

视听文本机器翻译质量测评和译后编辑研究——以《畅游赣鄱——水文化英文之旅》慕课字幕翻译为例

机器翻译给人们带来了便利,同时促成了译后编辑的诞生。本文以《畅游赣鄱——水文化英文之旅》慕课字幕文本为研究样本,在MQM(Multidimensional Quality Metrics)质量评估框架基础上提出视听文本机器翻译产品质量评估标准,量化其错误值,测试机器翻译质量,并由此提出译后编辑的三级模式:轻度译后编辑(Light Post-editing, LPE)、中度译后编辑(Moderate Post-editing, MPE)和重度译后编辑(Full Post-editing, FPE)。

面向社区图书馆的音像资源阅读推广活动研究——以北京市社区图书馆为例

音像资源作为馆藏非书资料之一,在图书馆阅读推广活动中扮演着重要角色。社区图书馆作为现代公共文化服务体系的公益性基础节点,承担着向居民个体提供均等化服务的重要职责。论文指出了音像资源社区图书馆阅读推广的理论导向,诠释了其在社区图书馆阅读推广中的价值体现,结合调研揭示了推广问题症结,然后有针对性地提出了推广策略。通过建立需求调研机制、塑造适合的阅读空间、加强音像资源配置、开通自助借还服务、打造线下活动品牌、搭建网络阅读推广平台六项可行性、有效性的实践举措,发挥音像资源在基础公共文化空间引领居民文化自信的作用。

大学英语教学语境创设路径研究与实践——以大学英语视听说课程为例

在大学英语视听说教学中,有效的视听输入直接影响学生英语输出的效果。语境创设为视听输入创造条件,提供可理解的、有效的英语语言输入,继而能促成学生更准确、自然地输出英语。基于输入输出假设,探究语境创设的有效路径,通过输入疏通、输出鼓励、及时调控、巧妙引导、恰当激励,促进英语学习者更准确、更自然地输出英语,提升其英语视听说能力。