词汇表示学习研究进展

A Review of Word Representation Learning

词汇语义表示是自然语言理解的基础。传统的基于语义词典的编码表示构建成本高昂,而独热表示又存在高维稀疏等缺点。词汇的分布式表示将词汇映射为低维稠密的实值向量,能有效捕捉词汇间的语义关联,是当前主流的表示技术。本文从数据特征、学习目标和优化算法三个方面,对现有的词汇表示学习方法进行了全面深入的分析,重点介绍了这些方法的理论基础、关键技术、评价指标及应用领域。此外,本文还总结了该方向面临的主要挑战以及最新研究进展,并对词汇表示学习未来的发展方向做了展望。

Word representation that reflects semantic meaning is fundamental to natural language understanding tasks. The traditional method of encoding a word through a semantic dictionary is impractical due to the high construction cost, and one-hot representation suffers from various defects, such as high dimension and data sparsity. Distributed word representation,which projects the words into vectors in a low-dimensional real-valued space, can capture the semantic relatedness between the words and has been widely used in many NLP tasks. In this paper, we present an in-depth study of word representation learning methods from the perspectives of input data, learning objectives, and optimization algorithms, focusing on the theoretical basis, key techniques, evaluation methods, and application fields. We then summarize the main challenges and the latest advances in this research field, and we finally discuss possible future work in the field.