基于话题翻译模型的双语文本纠错

TOPICS TRANSLATION MODEL-BASED BILINGUAL TEXT ERRORS CORRECTION

近年来,随着信息全球化的影响,社交网络文本上的多语言混合现象越来越普遍。许多中文文本中混杂着其他语言的情况已很常见。绝大多数现有的自然语言处理算法都是基于单一语言的,并不能很好地处理多语言混合的文本,因此在进行其他自然语言处理任务之前对文本进行预处理显得尤为重要。面对网络文本语义空间双语对齐语料的匮乏,提出一种基于话题翻译模型的方法,利用不同语义空间的语料计算网络文本语义空间的双语对齐概率,再结合神经网络语言模型将网络混合文本中的英文翻译成对应中文。实验在人工标注的测试语料上进行,实验结果表明,通过不同的对比试验证明文中的方法是有效的,能提升翻译正确率。

Along with the globalisation of information in recent years,multilingual mixing phenomena have become increasingly popular in social networks texts. It is quite common in Chinese texts that other languages are mixed. Since most of the existing natural language processing algorithm is the monolingual task-based,the multilingual mixed text can＇t be well processed,therefore it is crucial to pre-process the text before carrying out other natural language processing tasks. For the lack of the corpus of bilingual alignment in network text semantic space,we proposed a topics translation model-based method,it calculates the probability of bilingual alignment of network text semantic space using the corpus in different semantic spaces,then incorporates neural network language model to translate the English in mixed network text to corresponding Chinese text. The experiment was set on a manual labelled test corpus. Experimental result indicated that through different comparative experiments it was proved that the proposed approach was effective and was able to improve translation accuracy.