交集型歧义字段切分方法研究

Study of Segmentation Strategy on Ambiguous Phrases of Overlap Type

本文通过动态建立独立成词能力频次库的方法以及基于词语 /词性搭配的规则库 ,对交集型歧义字段进行处理 ,大大提高了切分正确率。在 4万语料的开放测试中 ,交集型歧义字段的切分正确率可达 98%以上。

This paper presents a segmentation strategy on ambiguous phrases of overlap type with rules based on the independent wording ability frequency and collocating of words & parts of speech, which improves the accuracy of segmentation greatly. In an open test of a Chinese corpus with 40000 characters, the accuracy of segmentation for ambiguous phrases of overlap type reached 98%.