汉语分词技术综述

Chinese word segmentation system research

分词是中文信息处理的基础,在汉语文本分类、文献标引、智能检索、自然语言理解与处理等应用中,首先都要对中文文本进行分词处理。从分词的基本理论出发,对近年来的汉语分词的研究方法与成果进行了综合论述,分析了现有分词方法的特点,提出了把神经网络和专家系统结合起来建立集成式汉语自动分词系统的构想。

Word segmentation is the basis of Chinese language processing, which is widely applied in text classification, literature index, intelligent retrieval and natural language processing. Based on the theory of the Chinese word segmentation, an overview of the Chinese word segmentation technology is presented,the existent Chinese Word Segmentation System is analyzed and an automatic word segmentation algorithm derived from artificial neural network and expert system is put forward.