基于DOM树的可适应性Web信息抽取

Adaptive Web Information Extraction Based on DOM Tree

Web信息抽取通常采用的是一种归纳学习方法,从给定的训练样本网页中学习到抽取规则,这种方法虽然能够准确地抽取出信息,但是当网站的模版发生改变后,必须重新获得抽取规则,因而这种抽取器的维护成本比较高,可适应性差。提出一种新的可适应性Web信息抽取方法,该方法首先通过聚类方法获取商品在网页中频繁出现的关键词组,然后利用网页的DOM树结构来确定包含这些关键词的信息块,从而实现Web信息的自动抽取。对大量商业网站进行信息抽取的实验表明,该算法不仅能有效抽取出商品信息,而且是一种与站点结构无关的可适应性信息抽取方法。

Many Web information extraction methods are related to wrapper induction. It extracts the items by the rules learnt from the Web pages used for training. Although it can get the information accurately, it is hard to be maintained when the template of the Web site is changed,as it needs to learn the rules again. In our research,we put forward a new adaptive Web information extraction. It determines the block which contains all information about the merchandise by u- sing the keywords of a certain topic,which is based on DOM tree structure. The experiments on a great amount of Web pages show that our method can not only extract the information efficiently, but also is irrelevant to the site structure, which can be widely used for many different Web information extractions.