基于HMM的百科辞典文本中句子的知识点分类

A HMM Based Topic Categorization of Sentences in Encyclopedia Texts

在百科全书条目文本中,往往以几个不同的侧面说明一个条目,一个侧面构成有关这一条目的一个知识点。知识点是一般文本处理领域中话题这一概念在百科全书文本中的具体表现。属于同一个类型的条目文本中总是重复出现有限的几个知识点,这些知识点构成了有关该类型的知识点集合,并且这些知识点在该类型不同的条目文本中的分布有较强的规律性。在条目文本中识别出对应的知识点序列是百科全书中知识提取过程的重要一环。该文提出了一种基于隐马尔科夫(HMM)模型的方法,利用知识点在条目文本中的转移规律以及知识点的词特征分布来判断每个句子的知识点类别。实验表明这个方法能取得较好的结果,在《中国大百科全书》地理卷的中国县市类型的条目文本中的实验结果的正确率为91.8%。

There is a certain common text structure shared by the articles dealing with the same kind items of encyclopedia.In other words,there are several common topics to appear in most of these texts,and the sequences of the topics in these texts are similar to each other.In the case of articles dealing with Chinese City,every sentence is covered by one of the common topics,like Area and Population,Clime etc.Automatic categorization of each sentence according to common topics will be very useful in encyclopedia knowledge retrieval.This paper presented a HMM based method of sentence categorization according to common topics,which utilizes the topic transitional information.Experiment shows that this method can provide acceptable performance,the precision of the sentence categorization in the Chinese City articles is 91.8%.