基于自然语言网络教学答疑中无词典分词算法的研究

Research on segmentation algorithm without dictionary based on natural language question answering system

提出了一种面向网络答疑系统的无词典分词方法.该方法用统计的手段从大规模未进行任何切分的领域语料中获取算法所需的参数,并结合一定的规则进行分词.该算法具有自学习的能力,适应性强,只要改变训练所用的语料,就能切分出不同领域的词.实验结果表明,该分词方法有较高的召回率和精度.

A segmentation algorithm without dictionary based on network-oriented natural language question answering system is proposed. This algorithm can acquire the necessary parameters from large-scale and being not cut Off field language material by statistical means, and cut off word with certain rules（ maximum pseudo-ambiguity field）. This algorithm has the ability of self-learning and good adaptability. If language material is changed,this algorithm can cut off different fields of word. The experimental results show that the segmentation algorithm improves recall and accuracy of cut off word.