基于可比语料库的双语词典抽取方法比较研究

Comparison of Approaches for Bilingual Lexicon Extraction from Comparable Corpora

双语词典是一种重要的语言资源,但现有的基于可比语料库的双语词典抽取方法在体系结构、所依赖的基础性资源等方面差异较大,这使得在统一的实验条件下对各种算法进行比较变得很困难.因此,目前的研究工作多选择将性能评测任务限定在很狭小的范围内,缺乏统一的评测结果给双语词典抽取任务的发展和算法的选择带来一定困难.为解决上述问题,选取并实现了四种代表性的双语词典抽取方案,在统一的测试数据集上进行比较研究.在比较研究中,我们重点揭示了词典抽取任务中几种关键因素如语料库大小、训练词典大小等对各算法性能的不同影响程度.本文的结论对今后相关工作中的实验设计、性能比较与算法选用都具有重要的理论意义和实践价值.

The bilingual lexicon is an important language resource. Current methods of lexicon extraction from comparable corpora differ in architecture and basic resources they rely on,which makes it difficult to make comparisons among various algorithms. Therefore, existing research work has performed the evaluation in a narrow range. The lack of unified evaluation results brings certain difficulty to the development of bilingual lexicon extraction task and the choosing of algorithms. To resolve above problems, this paper has chosen and implemented four representative methods of bilingual lexicon extraction,and makes a comparative study on unified data sets. It has mainly revealed the impact of such key factors as corpus size and training dictionary size on lexicon extraction task. The conclusion of the paper has great theoretical significance and practical value for the experimental design, performance comparison and algorithm choosing in the field of lexicon extraction from comparable corpora.