基于扩展标记树的网页正文抽取

Content Extraction of Web Page Based on Extended Label Tree

本文给出了一种基于扩展标记树的网页正文抽取方法,通过构建网页扩展标记树,实现对网页的清理和抽取辅助信息的完善,并设置节点坐标定位节点位置;以构成正文内容的文本节点作为正文区域标志,挑选具有最大文本覆盖范围的近邻文本节点集,并进行修正形成正文区域;通过近邻优先遍历算法,实现标题节点的定位和附加属性的抽取。实验结果表明:该方法可以实现常规文章类网页的高精度抽取,并具有良好的适应性。

A content extraction method based on extended label tree is proposed.Web page cleaning and auxiliary information for extracting purpose are realized,and the coordinates of position are also set during the construction phase of extended label tree.Text nodes are regarded as the identifiers of the content region,then,the neighbor text node set with maximum coverage is selected and revised to form the final content region.Through the neighbor first traversal algorithm,the title node is located and additional properties are extracted.Experimental results show that the proposed method can achieve high-precision for common article page extraction and has good adaptability.