从自动分词角度看先秦与现代汉语词汇区别

Differences between the Pre-Qin Chinese Words and the Contemporary Chinese Words from the Perspective of Automatic Segmentation

自动分词是古今汉语信息化所面临的共同课题。但古今汉语存在较大区别,现代汉语分词方法难以直接应用于古代汉语分词。为更好地借鉴现代汉语分词方法,探索符合先秦汉语的分词方法,从语言开放度、语言发展的表现及阶段、复音词的分布及构成、高频词的分布四个方面对先秦汉语与现代汉语的词汇进行了比较。发现先秦汉语既有区别于现代汉语的分词难点,又有独特的分词优势。

Word segmentation is the same problem of the pre-Qin Chinese and the contemporary Chinese information disposal. The methods of contemporary Chinese automatic segmentation can＇t be applied to pre-Qin Chinese automatic segmentation because of the difference of the two languages. The paper compares the pre-Qin Chinese words with the contemporary Chinese words from four angles： degree of language opening, manifestation and stage of the language de- velopment, distribution and constitution of the polysyllabic words, and distribution of the high-frequency words. It is found that there are not only distinctive difficult points but also distinctive merits for pre-Qin Chinese automatic segmentation.