基于视觉画面的空间音频合成及电影行业应用研究

Research on spatial audio synthesis and its application on film industry based on visual contents

新时代电影强国建设对电影技术的发展提出了新要求,创新视听算法和更优模型的自主研发成为人们关注的重点。部分早期影片存储时仅包含单声道音频,不能提供良好的听觉体验。为了解决这个问题,本文提出了一种利用视觉画面和已有单声道音频合成空间音频的方法,重构音频中的空间信息。该方法首先对单声道视频进行声源分离和定位,将其拆解成多个单声源及其位置信息。之后再对其进行空间音频重构,得到与视觉位置信息相一致的更加逼真的音频。实验结果表明,本文方法可以得到与视觉位置信息相一致的更加逼真的空间音频,给用户带来更好的沉浸体验。

The construction of a vibrant and energetic film country in the new era has put forward new requirements for the development of film technology.People pay more attention on innovating audiovisual algorithms and independent research and development of better models.Some early films were stored with only mono audio,which fail to provide an immersive auditory experience.To this end,this paper proposes a novel method of synthesising spatial audio,which reconstructs spatial information in the audio by leveraging visual contents and the given mono audio.This method first performs sound source separation and localisation on mono video and disassembles the mono video into multiple mono sources and their corresponding location information.Then,spatial audio reconstruction is performed to obtain more immersive audio consistent with visual contents.The experimental results showed that the proposed method can produce more realistic spatial audio synchronised with visual information,providing users with a more immersive experience.