多模态机器学习视域下字幕翻译的生成机制

The Generative Mechanism of Subtitling under Multimodal Machine Learning

字幕翻译不是纯文本的翻译,而是一种涵盖多模态的翻译类型。然而,由于相关研究大多着眼于符号和文本,字幕翻译并未跳出语言学的认知范畴。为此,本文借助多模态机器学习这一信息技术,将字幕翻译过程中所涉及的模态信息化,并根据多模态机器学习的原理,对字幕翻译的生成机制进行了剖析。本文具体探讨了字幕文本、声音和图像这三大模态的转化+协同表征、对齐和融合+联合表征机制,旨在完善字幕翻译的研究方法,为字幕翻译提供新的理论指导和技术支持。

Subtitling,unlike pure written text,is the one that involves multiple modalities.However,since most of the related studies focus on symbol and written text,subtitling has been confined to the cognitive category of linguistics.Therefore,this paper resorts to multimodal machine learning,an information technology,to informationize modalities involved in the subtitling process,and analyze the generative mechanism of subtitling in accordance with the principle of multimodal machine learning.This paper specifically investigates the translation plus coordinated representation,alignment and fusion plus joint representation mechanism of subtitle,voice and image,aiming to improve the research methods of subtitling and provide new theoretical guidance and technical support for subtitling.