基于分块的网页主题文本抽取

Extraction of Topical Text from Web Pages Based on Page Segmentation

根据网页文本信息的结构和内容特征,提出一种网页主题文本信息的抽取策略,将网页文档表示为DOM标签树的形式,然后根据Web页面的结构特征进行内容块的分割,以网页的文本内容特征为依据识别链接型和主题型内容块,并提取主题型网页的文本信息块。实验结果表明:基于分块的方法有效地实现了链接型和主题型网页的分类,并准确地完成主题型网页的文本信息块的抽取任务,是一种简单、准确的网页信息抽取方法。

According to the structure and content features of Web pages,a method is proposed,which is applied to extract topical text information in Web pages. Firstly,the Webpage is transformed to the form of DOM Tree, then divided into blocks based on the structural features. Secondly, content blocks are classified into link class or topic class using the content features. Finally,the topical text in topical Web pages is extracted. The experimental result indicates that this method has achieved the classification and extraction of Web pages in simple way ,but effectively and accurately.