Integrating Audio-Visual Features and Text Information for Story Segmentation of News Video

Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The proposed approach detects the topic-caption frames, and integrates them with silence clips detection results, as well as shot segmentation results to locate the news story boundaries. The integration of audio-visual features and text information overcomes the weakness of the approach using only image analysis techniques. On test data with 135 400 frames, when the boundaries between news stories are detected, the accuracy rate 85.8% and the recall rate 97.5% are obtained. The experimental results show the approach is valid and robust.

A Review on Audio-visual Translation Studies

This paper is dedicated to a thorough review on the audio-visual related translations from both home and abroad.In reviewing the foreign achievements on this specific field of translation studies it can shed some lights on our national audio-visual practice and research.The review on the Chinese scholars’ audio-visual translation studies is to offer the potential developing direction and guidelines to the studies and aspects neglected as well.Based on the summary of relevant studies,possible topics for further studies are proposed.

Audio-visual emotion recognition with multilayer boosted HMM

Emotion recognition has become an important task of modern human-computer interac- tion. A multilayer boosted HMM （ MBHMM ） classifier for automatic audio-visual emotion recognition is presented in this paper. A modified Baum-Welch algorithm is proposed for component HMM learn- ing and adaptive boosting （AdaBoost） is used to train ensemble classifiers for different layers （cues）. Except for the first layer, the initial weights of training samples in current layer are decided by recognition results of the ensemble classifier in the upper layer. Thus the training procedure using current cue can focus more on the difficult samples according to the previous cue. Our MBHMM clas- sifier is combined by these ensemble classifiers and takes advantage of the complementary informa- tion from multiple cues and modalities. Experimental results on audio-visual emotion data collected in Wizard of Oz scenarios and labeled under two types of emotion category sets demonstrate that our approach is effective and promising.

The Audio-Visual Performance Highlighted Craze in Chicago During Chinese New Year

February 10 (US Central Time), 2019, China National Peking Opera Company (CNPOC) and the Hubei Chime Bells National Chinese Orchestra presented a fantastic audio-visual performance of Chinese Peking Opera and Chinese chime bells for the American audience at the world s top-level Buntrock Hall at Symphony Center.

Research on National Identity Based on National Audio-Visual Works: Taking Inner Mongolia as an Example

Mongolian audio-visual works are an important carrier of exploring the true significance to this national culture.This paper believes that the Mongolian people in Inner Mongolia constantly enhance the individual sense of identity to the overall ethnic group through the influence of film and television and music,and on this basis constantly evolve a new culture in line with modern and contemporary life to further enhance their sense of belonging to the ethnic nation.

Application of Task-based Teaching Method to College Audio-visual English Teaching

Based on the current situation of college audio-visual English teaching in China, this article points out that the avoidance in class is a serious phenomenon in the process of college audio-visual English teaching. After further analysis and combination with the characteristics of college English audio-visual teaching in China, it puts forward the application of task-based teaching method to college audio-visual English teaching and its steps, attempting to alleviate the avoidance phenomenon in students through task-based teaching method.

Prioritized MPEG-4 Audio-Visual Objects Streaming over the DiffServ

The object-based scalable coding in MPEG-4 is investigated, and a prioritized transmission scheme of MPEG-4 audio-visual objects （AVOs） over the DiffServ network with the QoS guarantee is proposed. MPEG-4 AVOs are extracted and classified into different groups according to their priority values and scalable layers （visual importance）. These priority values are mapped to the 1P DiffServ per hop behaviors （PHB）. This scheme can selectively discard packets with low importance, in order to avoid the network congestion. Simulation results show that the quality of received video can gracefully adapt to network state, as compared with the ‘best-effort＇ manner. Also, by allowing the content provider to define prioritization of each audio-visual object, the adaptive transmission of object-based scalable video can be customized based on the content.

意境美学与气氛美学的跨文化互释

在全球化语境下,作为中国美学之独特形态的意境美学,与西方晚近兴起的气氛美学之间存在颇多可比照阐发、交流互鉴之处。在主客关系上,意境美学与气氛美学都打破了传统美学主客二分的思维方式,分别以“心-物”模式和“身-物”模式致力于审美主体与对象之间的间性关系的建构;不过意境美学融入与运用了综合的审美心理机能,侧重审美活动中精神性体验,将气氛美学中建立在感知学之上的“自然之物”进行了抬升和跨越,故作为意境美学基本单位的意象可以视为气氛之物的形而上之维。正是由于以上关联之处,一方面通感联觉是两种美学重要的审美生理-心理基础之一,另一方面作为审美范畴的“意境”和“气氛”都处于不断生成的状态之中,因而审美主体获得的美感体验都是历时性的情感体验,是一种“绵延”的美感。同时,意境美学和气氛美学都是在空间中进行审美,共同关注意境与气氛所营造的“虚实空间”。不仅如此,在情景创设、情调营造等方面,意境美学与气氛美学又可落实为一种共通的审美创造方案。基于以上事实,在意境美学与气氛美学之间进行广泛而深入的对话,既可为中西美学在交流互鉴中实现创新融合提供案例,还可在一定程度上激活中国古典美学的当代价值。

具身的诗意:跨专业通感设计的关联主义教学实验

本文介绍的是笔者近几年在跨专业与跨感官交叉设计教学中的探索。以“具身通感”实现“诗意体验”为设计教学目标,课程实验如何基于数字时代的关联主义理论,将不同专业背景、知识领域、工具路径和组织方式建立深度关联,培养学生在认知、体验与设计实践中具有高度自主的学习能力,尝试一种以知识关联与组织协同为核心的新型设计教学模式。

新媒体时代下视觉传达设计中的通感体验

目的 -通过研究分析新媒体技术加持下的通感体验和视觉传达设计,将这两者有机结合,以达到传达信息的最佳效果,给受众更好的体验。方法 -本文分析了新媒体时代下通感体验的表达基础和表达形式,以及视觉传达设计因媒介转变衍生出的创新设计形式,并以此为立足点探讨了新媒体时代下视觉传达中通感体验的设计方法。结果 -在新媒体时代下,视觉传达设计想要更加快速有效地传递,必须与多个感官表达相关联,多层次地调动受众的感官机能,更完美地表达出所要传递的信息。

Self-supervised Learning for Speech Emotion Recognition Task Using Audio-visual Features and Distil Hubert Model on BAVED and RAVDESS Databases

Existing pre-trained models like Distil HuBERT excel at uncovering hidden patterns and facilitating accurate recognition across diverse data types, such as audio and visual information. We harnessed this capability to develop a deep learning model that utilizes Distil HuBERT for jointly learning these combined features in speech emotion recognition (SER). Our experiments highlight its distinct advantages: it significantly outperforms Wav2vec 2.0 in both offline and real-time accuracy on RAVDESS and BAVED datasets. Although slightly trailing HuBERT’s offline accuracy, Distil HuBERT shines with comparable performance at a fraction of the model size, making it an ideal choice for resource-constrained environments like mobile devices. This smaller size does come with a slight trade-off: Distil HuBERT achieved notable accuracy in offline evaluation, with 96.33% on the BAVED database and 87.01% on the RAVDESS database. In real-time evaluation, the accuracy decreased to 79.3% on the BAVED database and 77.87% on the RAVDESS database. This decrease is likely a result of the challenges associated with real-time processing, including latency and noise, but still demonstrates strong performance in practical scenarios. Therefore, Distil HuBERT emerges as a compelling choice for SER, especially when prioritizing accuracy over real-time processing. Its compact size further enhances its potential for resource-limited settings, making it a versatile tool for a wide range of applications.

Cogeneration of Innovative Audio-visual Content: A New Challenge for Computing Art

In recent years,computing art has developed rapidly with the in-depth cross study of artificial intelligence generated con-tent(AIGC)and the main features of artworks.Audio-visual content generation has gradually been applied to various practical tasks,including video or game score,assisting artists in creation,art education and other aspects,which demonstrates a broad application pro-spect.In this paper,we introduce innovative achievements in audio-visual content generation from the perspective of visual art genera-tion and auditory art generation based on artificial intelligence(Al).We outline the development tendency of image and music datasets,visual and auditory content modelling,and related automatic generation systems.The objective and subjective evaluation of generated samples plays an important role in the measurement of algorithm performance.We provide a cogeneration mechanism of audio-visual content in multimodal tasks from image to music and display the construction of specific stylized datasets.There are still many new op-portunities and challenges in the field of audio-visual synesthesia generation,and we provide a comprehensive discussion on them.

“色可暖,话可凉”:英语温觉词的通感模式及其认知机制

温觉是触觉的一种,属于人类最基本的体验,其词汇通感模式是隐喻研究的重要问题。在体认语言学理论视角下,对英国国家语料库(BNC)中基本和一般温觉词(“cold/hot/cool/warm”;“fiery/burning/chilly/icy”)的通感使用频率与实例进行统计和分析,可以探究其映射规律和生成机制。研究发现,英语温觉词在整体上均能通感到视、听、味、嗅四种感觉域,通感率排序从高到低为视觉>听觉>味觉>嗅觉;基本温觉词通向味嗅觉的比例较低,部分一般温觉词无味嗅觉域上的通感表达。结果表明,通感是人类体验世界的基础,其映射模式和生成机制契合“现实—认知—语言”基本体认框架:某一感官信息通过意象图式在现实和认知之间交互形成概念,再由认知加工将概念转化为描述其他感官的语言表达。然而,受词汇编码强度和生理特性等因素的影响,具体词汇通感模式略有差异。

音乐联觉机制在抑郁症患者中的应用

目的探讨音乐的联觉效应在护理抑郁症患者中的应用。方法将2021年3月—2022年3月郑州中医院脑病一科收治的52例抑郁症患者随机分为2组。一组是常规药物治疗的对照组,实施传统护理方式;另外一组为音乐联觉机制与药物治疗相结合的观察组,实施多元化护理方式。以住院时间为周期,以2周为节点使用贝克抑郁量表第二版(BDI-Ⅱ)评估抑郁症状。护理人员对观察组和对照组的睡眠质量进行检测分析。结果干预后,2组患者的BDI-Ⅱ评分均低于干预前,且观察组低于对照组,差异具有统计学意义(P<0.05)。观察组比对照组的总睡眠时长、觉醒次数少、睡眠潜伏期短、睡眠效率高,且入睡后觉醒时间短。结论采用多元护理与音乐联觉机制相结合的治疗方法,治疗效果比单纯常规治疗效果明显,音乐联觉机制在治疗抑郁症患者的过程中有显著作用。

元宇宙视域下虚拟影响者的立体性通感体验设计研究——以短视频平台传播为中心

在元宇宙趋势盛行的当下,随着短视频平台应用率的大幅上升,新型虚拟形象应用也成为短视频平台的聚焦热点。通过分析比较短视频平台不同功能类型的虚拟影响者间感官体验元素的应用,探讨虚拟影响者的通感体验设计原则,解决虚拟影响者在短视频平台中视觉、互动、情感层面的设计问题。基于通感体验理论下借助案例分析研究方法,对短视频平台虚拟影响者进行对比分析。总结归纳出偶像型虚拟人、服务型虚拟人、超写实虚拟人的通感体验设计趋势,借助这3种类型提升虚拟形象目标受众的用户体验,有助于实现虚拟影响者在短视频媒介的跨界传播。

基于5G/5G-A的全要素车联网行业综合应用实践

由于5G技术的演进为智能网联汽车开展各类自动驾驶,尤其是高级别自动驾驶所需的全要素信息综合应用提供了新的解决思路和方案,因此从技术演进、解决方案、性能测试以及应用孵化4个方面阐述5G/5G-A最新技术在车联网领域的相关性能测试和应用试点,并提供全要素车联网综合解决方案。

异质同构,传递基因——文化景观视角的历史城区更新形式探讨

从人文地理学的文化景观角度切入,分析城市形态的文化内涵与空间视觉特征的关联性,以及其在城市历史地段更新行动中指导空间形式有依据地渐变演进的可能性,并借助心理学认知原则对这一过程中文脉基因的可识别性进行论证,提出以“异质同构”的形式更新策略来响应城市历史意象和历史信息的延续和维护。

Synesthesia, Experiential Parts, and Conscious Unity

Synesthesia is the ＂union of the senses＂ whereby two or more of the five senses that are normally experienced separately are involuntarily and automatically joined together in experience. For example, some synesthetes experience a color when they hear a sound or see a letter. In this paper, I examine two cases of synesthesia in light of the notions of ＂experiential parts＂ and ＂conscious unity.＂ I first provide some background on the unity of consciousness and the question of experiential parts. I then describe two very different cases of synesthesia. Finally, I critically examine the cases in light of two central notions of＂unity.＂ I argue that there is good reason to think that the neural ＂vehicles＂ of conscious states are distributed widely and can include multiple modalities. I also argue that some synesthetie experiences do not really enjoy the same ＂object unity＂ associated with normal vision.

基于“联觉-通感”层次分析的沉浸式展陈交互设计研究

为了把沉浸式展陈中的多感官交互上升到情感交互,发挥出数字媒介艺术的优势,在分析沉浸式展陈交互设计现状的基础上,从多感官整合的交互感知建设,“联觉”效应下的交互认知赋意,情感共鸣下的“通感”语境创建三个方面探究了交互设计中“联觉-通感”层次模型,并以“新艺维工业设计”产品展陈交互设计实践为例,验证了基于“联觉-通感”层次分析的交互设计方法能够较好地改善沉浸式展陈设计中人的交互体验,增强人对企业文化内涵的理解。

英语联觉音学习意义与方法研究——以sn-所构词汇为例

针对现阶段联觉音语素应用不足、词汇学习效果不佳的窘境,文章引入sn-结构示例,简要阐述其隐喻性、象征性内涵,明确其在“口鼻”相关行为中的表现作用,深入分析联觉音学习的价值、意义,同时指出其融合应用可以显著降低英语词汇理解、接受难度,提升学习者对构词逻辑、规律的认知,帮助其拓宽词汇学习视野、思路。在此基础上归纳可用的联觉音学习方法,提出了挖掘形义连接关系、关注音义转换逻辑、建立文化认知视角等系列改进举措。