视觉—语言—行为:视觉语言融合研究综述

Vision-language-action:survey of integration of vision and language

通过语言给予智能体指示使其完成通用性的任务是人工智能领域的愿景之一。近年来有越多越多的学者试图通过融合计算机视觉与自然语言处理领域的相关技术以期实现此目标。为了及时跟进相关领域的研究,把握视觉与语言融合方向前沿,对于视觉—语言—行为最新进展进行综述。首先简单介绍了融合视觉与语言的弱耦合尝试,之后重点综述了视觉—语言—行为这一最新方向相关的视觉语言导航、具身问答及其相似研究,最后总结了制约此领域发展的关键问题及可能的解决方案。

The idea that might be able to give general and verbal instructions to a agent and have at least a reasonable probability that it will carry out the required task is one of the long-held visions of robotics and artificial intelligence(AI).It has noticed that more and more scholars in recent years have tried to this target using the latest advances in the field of computer vision and natural language progressing.In order to follow up the research in related fields and grasp the frontier of the fusion of vision and language,this paper reviewed the latest progress of visual-language-behavior.It first introduced the weak coupling experiment of fusion between vision and language,then focused on the vision-and-language navigation,the embodied question answering and their similar research,and finally looked forward to the future development trends in this field.