基于半监督的汉缅双语词典构建方法

Semi-supervised Chinese-Burmese Bilingual Dictionary Construction

汉缅双语词典是开展机器翻译、跨语言检索等研究的重要数据资源。当前在种子词典的基础上使用迭代自学习的方法在平行语料中抽取双语词典取得了较好的效果,然而针对低资源语言汉语-缅语的双语词典抽取任务,由于双语平行资源匮乏,基于迭代自学习的方法不能得到有效的双语词向量表示,致使双语词典抽取模型准确度较低。研究表明,可比语料中相似词语往往具有相似的上下文,为此,该文提出了一种基于半监督的汉缅双语词典构建方法,通过利用预训练语言模型来构建双语词汇的上下文特征向量,对基于可比语料和小规模种子词典的迭代自学习方法得到的汉缅双语词汇进行语义增强。实验结果表明,该文提出的方法相较于基线方法有明显的性能提升。

Chinese-Burmese bilingual dictionary is an important data resource for research on machine translation and cross-language retrieval, etc. At present, the iterative self-learning method based on small-scale seed dictionary has achieved good results in extracting bilingual dictionaries from parallel corpus. However, for low-resource languages like Chinese-Burmese bilingual dictionary extraction task, due to the lack of bilingual parallel resources, the method based on iterative self-learning can not get effective bilingual word vector representation, resulting in the low accuracy of bilingual dictionary extraction model. Recent studies suggest that similar words in comparable corpora often have similar contexts. Therefore, this paper proposes a semi-supervised method for constructing Chinese-Burmese bilingual dictionary. By using the pre training language model, the context feature vector of bilingual vocabulary is constructed. The Chinese-Burmese bilingual vocabulary obtained by the iterative self-learning method of comparable corpus and small-scale seed dictionary is semantically enhanced. The experimental results show that the proposed method has a significant improvement comparing with the baseline method.