基于语义的主题爬行策略

Semantic-Based Focused Crawling Approach

为使主题爬行能够充分利用资源的语义信息,提出基于语义的主题爬行策略.该策略利用领域本体刻画爬行主题,将本体语义映射到关键词表.通过定义断言集一致性扩展和域值关联推理任务,推演关键词间语义关系.在定义网页主题概念的基础上,结合本体推理方案提出主题概念的语义叠加效应模型.最后,利用主题概念的语义包含关系判定URLs抓取顺序.实验结果表明,该语义主题爬行策略在抓取收获率和爬行效率上优于现有同类方法,该方案有效、可行.

An approach of semantic-based focused crawling is proposed in order to use semantic resource efficiently. In this paper, a domain-ontology is used to describe the topic of Web crawling. Lexicon of the keywords list are mapped to ontology, and semantic of words are obtained through mapping. Inference services about assertion set expanding and domain-range relation are defined. The semantic relation among keywords can be inferred by inference services. At the same time, the definition of concept about Web page is given. A semantic computational model is proposed by combining inference services mentioned above. In the end, the order of URLs corresponding to their Web page is decided according to the subsumption of topic concepts. The result show that this approach is advanced in harvest-rate and crawling efficiency and is better than some classical algorithms.