一种基于双哈希二叉树的中文分词词典机制

A DICTIONARY MECHANISM FOR CHINESE WORD SEGMENTATION BASED ON DOUBLE HASH BINARY TREE

汉语自动分词是汉语信息处理的前提,词典是汉语自动分词的基础,分词词典机制的优劣直接影响到中文分词的速度和效率。详细介绍汉语自动分词的三种方法及五种词典机制,提出一种简洁而有效的中文分词词典机制,并通过理论分析和实验对比说明几种词典机制的效率差异来证明所提出分词词典机制的有效性。

Automatic Chinese word segmentation is the prerequisite for Chinese information processing,and dictionary is the basis of automatic Chinese word segmentation.Whether a dictionary mechanism of word segmentation is good or bad directly affects the speed and efficiency of Chinese word segmentation.In this paper,three kinds of Chinese word segmentation means and five kinds of dictionary mechanisms are introduced in detail.In the end,a simple and effective dictionary mechanism for Chinese word segmentation is proposed.Furthermore,through theoretical analysis and experiment,we compare and illustrate the differences in efficiency of a couple of dictionary mechanisms,and use the conclusion to prove the effectiveness of our dictionary mechanism for word segmentation.