自动抽取web数据的树对齐算法

Automatic web data extraction based on tree alignment

针对从模板生成的网页中自动抽取web数据的问题,提出了一种新的树对齐算法.该算法能够确定输入网页的最大匹配结构.经过一系列的对齐操作之后,多棵树被合并成为一棵记录着合并前多个网页上的统计信息的合并树,树对齐算法可以发现合并树中的重复模式,在最可能内容块上构建包装器,并按照重复模式从网页上抽取数据.实验结果表明,该算法的抽取结果具有较高的准确性和良好的稳定性.

This paper proposed a new tree alignment algorithm for determining the optimal matching structure of the input web pages, in order to extract web data automatically. Based on the alignment, the trees were merged into one union tree whose nodes record statistical information obtained from multiple web pages. The algorithm detects repeating patterns on the union tree, and a wrapper built on the most probable content block and the repeating patterns extracts data from web pages. Experimental results showed that the proposed algorithm achieves high extraction accuracy and has steady performance.