基于Ontology的Web内容二阶段半自动提取方法

Ontology-Based Two-Phase Semi-Automatic Web Extracting

目前Web中的海量信息已经成为人们重要的信息来源 ,如何从大量半结构化或无结构的HTML网页中提取信息已成为目前的研究热点 .但是Web页面的初始设计目的是为了方便用户浏览 ,而不是便于应用程序自动处理 ,如何实现一个精确的、应用广泛的提取系统面临很多困难 .传统的方法可以粗略划分为基于交互产生的包装程序和自动生成的包装程序 ,但是基于交互产生的包装程序不具备普遍的应用性 ,基于自动生成的包装程序准确性不高 .该文提出了一种新的二阶段基于语义的半自动提取方法 ,在保证提取准确性的前提下 ,尽可能减少交互操作 ,同时随着参与网站的增加 ,逐步提高包装程序生成的自动化 .相对于目前的方法 ,该文方法同时考虑了包装程序提取结果的准确性和提取过程的应用普遍性 .其有效性在原型系统中得到验证 .应用该方法 ,已经成功提取了12 0万HTML页面 .

The massive information on the Web has become an important information source for people. How to extract information from semi-structured or unstructured HTML pages receives much attention. However, the original intention of web pages is not to be processed by application automatically, but to be browsed by users. It is difficult to design a precise web data wrapper with high applicability. Roughly, existing methods can be classified into interactive-based wrapper generation and automatically wrapper generation, but the former lacks applicability while the latter lacks the precision of extraction. This paper proposes a novel two-phase semi-automatically precise web extracting method. The method tries to reduce the interactive work in wrapper generation process as much as possible while it should maintain the precision of extraction result at the same time. In addition, with the increase of extracted web pages, the automaticity in the process will also be improved. Compared with the existing methods, the method proposed takes both the precision of query result and the applicability of wrapper into consideration. The method has been validated in authors' prototype, which has extracted 1,200 thousand web pages successfully.