基于自动句对齐的相似古文句子检索

Ancient Sentence Search Based on Sentence AutoAlignment in Parallel Corpus of Ancient and Modern Chinese

随着语料库语言学的兴起,基于实例的机器翻译(EBMT)得到越来越多的研究。如何快速准确地构建大规模古今汉语平行语料库,以及从大量的对齐实例(句子级)中检索和输入句子最相似的源句子是基于实例的古今汉语机器翻译必须解决的问题。本文综合考虑句子长度、汉字字形、标点符号三个因素提出了古今汉语句子互译模型,基于遗传算法、动态规划算法实现了古今汉语的自动句对齐。接着为古文句子建立全文索引,基于汉字的信息熵,本文设计与实现一种高效的最相似古文句子检索算法。最后给出了自动句对齐和最相似古文句子检索的实验结果。

Along with the Corpus Linguistics＇ prosperity and development, the research on Example Based Machine Translation （EBMT） has a flourishing prospect. In this area, two problems must he solved： 1） Constructing a large --scale parallel corpus with high accuracy and speed. 2） Searching the most similar sentence with the input sentence from the huge aligned examples. This paper aimed at EBMT between ancient and modern Chinese. First, a new translation model was built which takes the length of the sentence, character information and punctuation into account at the same time. Then, a new approach for aligning bilingual sentences automatically was proposed based on genetic algorithm and Dynamic Programming. Finally, a new similarity method was given based on Chinese characters＇ information entropy. Experimental results showed that our methods achieved good performance.