基于单字提示特征的中文命名实体识别快速算法

A Rapid Algorithm to Chinese Named Entity Recognition Based on Single Character Hints

近年来条件随机场(CRF)模型在自然语言处理中的应用越来越广泛。标准的线性链(Linear-chain)模型一般采用L-BFGS参数估计方法,收敛速度慢。本文在分析模型复杂度的基础上提出了一种改进的快速CRF算法。该算法通过引入小规模单字特征降低特征的规模,并通过在推理过程中引入任务相关的人工知识压缩Viterbi和Baum-Welch格搜索空间,提高了训练的速度。在中文863命名实体识别评测语料和SIGHAN06语料集上进行的实验表明,该算法在不影响中文命名实体识别精度的同时,有效地降低了模型的训练代价。

Conditional Random Fields （CRF） model becomes prevalent for sequential labeling tasks in the field of NLP. A general but slow optimization algorithm L-BFGS is commonly used in parameter estimation of CRF Model. In this paper, an improved algorithm is proposed to train CRF model more quickly. First, small scale character hint features are introduced to decrease the feature space. Then, a task-specific rule is applied to reduce search paths in Viterbi and Baum-Welch procedure. The experiments on China 863 program NER and SIGHAN 2006 corpora show that our schema saves training time significantly without performance drop.