视听跨模态交互作用的研究现状

Review of Audio-Visual Cross-Modal Interaction

人类在观察和感知外界事物时,需要从多个感官模态获取信息。大脑在获得这些信息以后,随即进行信息整合形成认知过程,进而获得认知结果,并指导后续行为。视听信息的交互作用在实际生活中极其常见,是人类认知活动中的重要组成部分,也因此成为近年来的研究热点。本文对视听交互作用的研究现状进行了总结,其中包括视听一致促进认知、视听不一致调整认知等,重点阐述了近些年来在单一视觉或听觉感官产生的主导作用这方面所取得的突出成果,并在此基础上展望了今后可能的研究方向。

Mankind collects the information from multi-sensory modalities when detecting and perceiving objects and events. Such multimodal inputs, after being achieved by the brain, are rapidly integrated to form cognitive processes, achieve perception, and finally guide people＇ s behaviors. Audio-visual integration is extremely common in daily life and plays,an important role in human cognitive activities. Therefore, audio-visual cross-modal interaction has become a hot topic in recent years. This paper gives an overview of audio-visual interaction and describes the achievement by focusing on the enhancement of cognition with congruent audio-visual stimuli, the adjustment of perception with incongruent visual-auditory information, and emphasizes the development of uni-sensory dominant mechanism, based on which the outlook of research directions is discussed.