基于CRF的中文组块分析

Chinese Text Chunking Based CRF

提出一种基于条件随机域模型的方法用于中文文本组块分析.该方法将中文组块分析转化为对每个词语赋予一个组块标注符号,再根据条件随机域对标注好的训练语料建立模型,从而预测测试语料中每个词语的组块标注符号.使用北京大学中文树库的测试结果为F1=85.5%,高于隐马尔可夫模型和最大熵马尔可夫模型.实验结果表明,条件随机域在中文组块识别方面有效,并避免了严格的独立性假设和数据归纳偏置问题.

A new method to solve Chinese text chunking was introduced as conditional random fields （CRF） model, by which Chinese text chunking transformed into labeling the words with their chunk tags and establishing a model for tagged corpus according to Conditional random fields so as to predict the chunk tag of each word. An F1 score of 85.5% is achieved by using the evaluation dataset of Chinese treebank of Beijing university, and obviously better than those of hidden Markov model and maximum entropy Markov model. Experimental results show that conditional random fields model is an effective way on Chinese text chunking and the strict Independence hypothesis and the label bias problem are avoided.