手语计算30年:回顾与展望

Thirty Years Beyond Sign Language Computing:Retrospect and Prospect

手语的自然语言处理是计算机学科中的一项重要任务.目前随着信息技术的飞速发展,以文本和语音为主要载体的传统语言计算的工作重点已从编码、输入方法和字音的研究逐渐转移到语法层面,并进入深度计算的阶段.然而手语信息处理却严重滞后,处于空白起步阶段.究其原因,主要是缺乏用于机器学习的具有一定规模的手语语料库资源,同时传统的语言计算技术也存在不足,这些都阻碍了手语机器翻译、手语问答系统、手语信息检索等信息处理的应用研究.该文首先阐述了手语计算与传统语言计算的本质差异在于空间建模,这种差异导致了前者核心任务是单信道与多信道转换,后者根本任务是消歧.从词法、句法、语义、语用、应用等层面对手语计算进行了回顾,重点介绍了手语机器翻译和分类词谓语计算,指出分类词谓语是手语计算的关键以及取得突破的切入点.从展望的角度,认为互联网时代体感设备的出现、认知神经科学的兴起、深度学习的进展等新技术为手语计算带来了新的机遇.将手语计算与传统语言计算进行比较,分析了手语计算的趋势和未来的研究方向,手语的认知计算是从手势的物理特征到语义表征的映射转换过程,其计算趋势是填补音韵特征、语义单元这样的中间步骤,避免直接从底层特征得到语义概念,关注在手语行为与语言特征的关系上进行机器学习,建立融合空间特征的统计学习模型.未来研究方向包括资源建设、文景转换、隐喻理解,其中文景转换有助于实现空间信息抽取,即物体的空间方向、位置等信息,结合知识库消除自然语言的模糊性,进而实现三维场景构建.指出手语计算正从萌芽期过渡到发展期,若取得重大突破,手语计算将扩展语言计算体系,推动人工智能的发展.

The natural language processing of sign language is an important task in the field of artificial intelligence and information processing.Currently,with the development of information technology,the focus on the information processing of spoken language and written language,is gradually shifting from the word coding and input method to the grammatical level,and then to depth computing.However,sign language information processing is seriously lagging behind and remains at the starting stage.The main reason for this situation is that no ready-made sign language corpus resources can be used for machine learning and deep learning.Sign language machine translation,sign language question-answering system,information retrieval and information processing cannot be applied because of the lack of research foundation.The essential difference between sign language computing and traditional language computing is spatial modeling and it leads to that the core task of sign language computing is to convert single-channel representation to multi-channel representation,while the fundamental task of the traditional language computing is the disambiguation of single-channel representation.From the lexical,syntactic,semantic,pragmatic,and applied levels,sign language computing is reviewed,and the sign language machine translation and classifier predicates in computing are emphatically introduced.Classifier predicates are the key of sign language computing,and it is the breakthrough point of sign language computing.New technologies,such as the emergence of somatosensory devices,the rise of cognitive neuroscience and the progress of deep learning,have brought new opportunities to sign language computing in the Internet age.From the perspective of outlook,sign language computing is compared with spoken language computing.The trend of sign language computing and the breakthrough points are analyzed.The cognitive computing of sign language has been regarded as a mapping conversion process from the physical characteristics of gestures to semantic representation.The trend of sign language computing is to fill these intermediate steps,such as phonological features and semantic units.It avoids the semantic concepts obtained directly from the underlying physical features,focuses on the machine learning on the relationship between sign language behavior and language features,and establishes the statistical learning model of fusion spatial features.These breakthrough points include resource construction,text-to-scene and metaphor understanding.Among them,the text-to-scene in the sign language is helpful to realize the spatial information extraction including spatial orientation,object position,and the ambiguity of natural language can be eliminated by combining with the knowledge base,so as the three-dimensional scene construction can be achieved and creates a breakthrough in understanding the spatial relationship and generating the virtual scene.It is pointed out that the sign language computing is from the embryonic period to the development period.Driven by the interdisciplinary,sign language computing may make a substantial breakthrough.The astonishing progress of traditional language computing has promoted artificial intelligence and human-computer interaction to develop further.If a series of problems about sign language computing can be solved in fields of theory,technology and engineering,it will greatly speed up the development of artificial intelligence and natural language processing.