基于词向量和条件随机场的领域术语识别方法

Domain-specific term recognition method based on word embedding and conditional random field

针对基于统计特征的领域术语识别方法忽略了术语的语义和领域特性,从而影响识别结果这一问题,提出一种基于词向量和条件随机场(CRF)的领域术语识别方法。该方法利用词向量具有较强的语义表达能力、词语与领域术语之间的相似度具有较强的领域表达能力这一特点,在统计特征的基础上,增加了词语的词向量与领域术语的词向量之间的相似度特征,构成基于词向量的特征向量,并采用CRF方法综合这些特征实现了领域术语识别。最后在领域语料库和Sogou CA语料库上进行实验,识别结果的准确率、召回率和F测度分别达到了0.985 5、0.943 9和0.964 3,表明所提的领域术语识别方法取得了较好的效果。

Domain-specific term recognition methods based on statistical distribution characteristics neglect term semantics and domain feature, and the recognition result are unsatisfying. To resolve this problem, a domain-specific term recognition method based on word embedding and Conditional Random Field （CRF） was proposed. The strong semantic expression ability of word embedding and strong field expression ability of similarity between words and term were fully utilized. Based on statistical features, the similarity between word embedding of words and word embedding of term was increased to create the feature vector, term recognition was realized by CRF and a series of features. Finally, experiment was carried out on field text and SogouCA corpus, and the precision, recall and F measure of the recognition results reached 0.9855, 0. 943 9 and 0. 964 3, respectively. The results show that the proposed method is more effective than current methods.