基于网页布局相似度的Web论坛数据抽取

Data Extraction from Web Forums Based on Similarity of Page Layout

Web论坛中蕴含着丰富的信息资源,充分利用这些信息资源依赖于论坛数据抽取技术。该文解决了从Web论坛抽取什么数据和如何抽取的问题,提出了一种基于网页布局相似度的Web论坛数据抽取方法,有效弥补了目前方法的自动化程度低,或准确率低的不足。该方法充分利用Web论坛网页布局结构上的特点,采用分级处理的方式,先识别出主题信息块、再利用待抽取数据的统计规律在主题信息块中完成抽取,整个过程不需要任何人工干预。实验结果表明,新方法对不同的BBS站点有很好的通用性,且具有较高的准确率和召回率。

Web forums contain a wealth of information resources. Making full use of these information resources re- lies on web forums data extraction technology. This paper solves the problems of what data should be extracted and how to extract from web forums by the proposed method based on the similarity of page layout. The method can ef- fectively avoid the disadvantages of current methods at low degree of automation or low accuracy. The method firstly recognizes the topic block by making full use of the special layout of the web forum pages, then extract data using rules from the topic block. Experimental results show that this method performs well in adjustability, precision and recall.