口译多模态加工中的手势认知效应研究

Investigating the Cognitive Effects of the Interpreter’s Gestures in Multi-Modal Processing

本文使用ELAN多模态话语分析软件对两段中国导演海外影片宣传的口译活动进行了分析。结果发现手势是口译加工中重要的显身特征、信息模态和认知资源,具体来说:1) 认知负荷更大时,手势数量更多,手势位置更高,手势更加快速而频繁,但主要手势类型都是节奏性手势;2) 口译认知加工中的手势、语言和副语言产品具有跨模态统一性;3) 口译质量更高的译员更倾向于模仿讲者手势,但手势类型可能发生变化。同时,手势数量也可能受到现场语境因素以及译员认知资源的影响。本研究构建的初步框架和分析工具有助于弥补口译手势多模态研究的不足,相关结论亦可促进基于询证的口译教学。

This research uses a multi-modal discourse analysis software, ELAN, to analyse two inter-preter-mediated events featuring Chinese film directors promoting their films to an international audience. It is found that gestures represent an important embodied information modality and cognitive resource in interpreting processing. Specifically: 1) With higher cognitive load, more gestures with higher positions are made, with faster frequency and pace, although beats remain the dominant gesture type for all speakers, including interpreters;2) Gestures align with the interpreter’s linguistic and paralinguistic products during the multimodal cognitive processing of interpreting;3) Interpreters achieve higher interpreting quality scores through more imitation of speaker gestures, although the imitator may change the gesture type. Meanwhile, contextual factors and availability of the interpreter’s cognitive resources may also impact on the use of gestures. The preliminary framework and analysis tools built in this research could fill some of the gaps in understanding the multi-modal nature of interpreting gestures. The initial findings could also promote evidence-based interpreter education.