汉字简繁文本智能转换系统中语言学问题分析

Linguistic analysis of the intelligent conversion system of simplified and traditional Chinese characters text

两岸汉字虽属同一系统,但在两岸语言生活中,当前还存在不少差异,大陆使用规范字,台湾使用"标准字",也就是通常所说的"简繁汉字",这给两岸日常生活和文化深度交流带来了一定困难。论文重点从文字、词汇和标点符号等方面论述了国家语委2012年所启动的简繁文本智能转换系统研发工作中所存在的语言学问题。尽管当前的语言学分析已足够细致,统计模型也已很完善,转换准确性也极高。但完全实现愿望是不可能的,还需要人工辨别和干涉。因此,在文字层面上怎样更好地利用规则,而在词汇层面上,怎样更好地利用统计模型,是今后进一步提高准确性的关键所在。

Chinese characters of both sides of Taiwan Strait are the same character system,but in the language situaion on both sides,there are still many differences.Mainland uses＂standardized word＂and Taiwan uses the＂standard character＂,commonly called ＂Simplified and Traditional Chinese characters＂,which has brought some difficulties on cultural depth of cross-strait exchanges on daily life.This paper focuses on the linguistic issue of the text,vocabulary and punctuation,which started in 2012 by the State Language Commission.Although the current linguistic analysis has been careful enough and statistical models have been perfect and the conversion accuracy is extremely high,the desire to achieve 100%is impossible.Therefore,in the literal level,how to better take advantage of the rules,and at the lexical level how to make better use of statistical models is the key to further improve the accuracy in the future.