基于Zookeeper构建准实时索引更新系统及其监控

Quasi–real-time index update system using Zookeeper and its monitoring

Mysql数据库表切换过程,需要保证两个相同表同时写入。Solr作为企业级的搜索框架,其自带的索引更新系统是增量更新,实时性差。为了满足互联网企业对数据的实时性要求,设计和实现了一款基于Zookeeper的能够准实时更新Solr索引和实现数据库双写的系统。用户需要提交配置信息到系统,当Mysql数据源发生内容变更时,系统就能够实时捕获,将变更内容经过数据转化并实时同步更新Solr索引。

Mysql database table switching process, it is needed to ensure that two same tables are written at the same time. As an enterprise-class search framework, Solr’s built-in index update system is incremental update with poor real-time performance. In order to meet the real-time requirements of Internet enterprises, a system is designed and implemented by using Zookeeper, which can quasi – real-timely update Solr index and realize database dual write. Users need to submit configuration information to the system. When the content of Mysql data source changes, the system can capture the changed content in real time, and update Solr index synchronously in real time through the data transformation.