基于Lucene的“农搜”并行索引技术研究

Research on Lucene Parallel Index Solution Based on Agricultural Search Engine

Lucene作为高度优化的倒排索引搜索引擎为搜索向垂直化和专业行业化发展提供了可能,打破了搜索的高技术壁垒。但在实际应用过程中遇到了两个主要问题:①随着被索引文件的增多,索引时间成线性增长,导致建索引的过程会影响搜索体验;②搜索服务器的硬件门槛导致无法实现分布式索引。本文采用多台PC同时建索引再合并索引的方法形成了一个可扩展的搜索引擎解决方案。极大地缓解了建索引给搜索带来的问题。

As a highly optimized inverted index search engine, Lucene enable the search to develop toward the vertical and professional services, with breaking the barriers of high search technology. However, the two main difficuhies are encountered in the course of practical applications. First, with the increasing numbers of indexed documents, linear growth of indexing time results in the impacts of the process of building index on the search experience. Secondly; the threshold of the search server hardware lead to failure of implementing distributed indexing. Through the method of merger indexing after many PC Units create index at the same time, a scalable search engine solution was found. It greatly ease the searching problem introduced in creating index.