Web文档中词语权重计算方法的改进

Improved approach to weighting terms in Web Text

以向量空间模型作为Web文本的表示方法,对传统的TF*IDF公式进行了改进。首先,结合Web文本中HTML标签的修饰功能,体现了特征词在Web文本结构中的位置信息;其次,以广义信息论为理论基础,引入了基于二次熵的互信息作为权重计算公式的一项,体现了单词的类区分能力。实验验证了该方法的可行性和有效性。

This paper uses vector space model as the description of the Web text,analyses and improves the traditional formula TF＊IDF.First,we explore the feature of the Web pages which are written in HTML and describe the situation information of the terms in Web text.Second,we use generalized information theory as the theory base to introduce the quadratic entropy mutual information into the formula.The experiment shows the feasibility and the validity of this method.