面向主题的垂直搜索引擎系统的研究与实现

Research and Implementation of Subject-Oriented Vertical Search Engine System

针对通用搜索引擎的信息量大、查询不准确、深度不够等问题,给出了面向主题的垂直搜索引擎的体系结构,设计了垂直搜索引擎系统的爬行策略,对系统核心信息采集模块运用了多线程技术及基于VSM的主题相关度判断算法进行主题网页爬行,并通过Lucene.Net的索引与检索技术建立系统的检索算法,实现了一个面向特定主题的垂直搜索引擎应用系统.实验测试结果表明,该系统具有较高的提取效率,其检索的准确率、召回率均大大高于通用搜索引擎,具有较好的实用价值和商业应用前景.

A general search engine usually suffers from returning to users too much unrelated information.To avoid these problems of general search engine,a subject-oriented vertical search engine is proposed in this paper followed by the introduction of the crawling strategy and the architecture of this vertical search engine.Some distinctive characteristics of this search engine include： the system runs its core modules on multiples threads;a topic-related determination algorithm based on VSM is implemented in the system.The search engine system as well as its core algorithms is built upon lucene.Net.Experimental results show that the accuracy and recall rate of this system are higher than general search engine.Therefore,it is practically valuable and can be applied into more real applications.