运用开源软件Logstash和ElasticSearch实现DSpace日志实时统计分析

Using Logstash and Elastic Search to Achieve Real-time Statistical Analysis of DSpace Logs

【目的】设计并实现DSpace日志实时统计分析系统,满足用户各种实时统计需求,弥补DSpace自带统计功能的不足。【应用背景】受DSpace系统自身设计的限制,其自带的日志统计功能单一,表现形式僵化,不能实现交互式统计分析。【方法】运用Logstash实时收集并分析DSpace日志,运用Elastic Search对分析后的日志进行索引,构建Query DSL查询调用Elastic Search的Java API实现不同的统计功能,并采用ECharts组件图形化展示结果。【结果】DSpace日志实时统计分析系统能够实现用户自定义时间区间统计条目、合集和社群的浏览排行,条目对象下载排行以及访问地区排行等。统计的结果可以以不同图表形式展现。【结论】运用Logstash和Elastic Search实现DSpace日志统计,不需要修改DSpace源代码,组件安装部署简单,实现人机互动式查询统计,统计结果快速且实时,结果展现形式多样。

[Objective] The real-time statistical analysis system of DSpace logs is designed and implemented to meet the different needs of users, and to make up for lack of DSpace＇s statistical functions itself. [Context] For the design limitations, the DSpace＇s statistical functions are simple, rigid form of expression, and can not achieve interactive statistical analysis. [Methods] Use Logstash to collect and analyze DSpace logs, and use ElasticSearch to index the logs Building QueryDSL to call ElasticSearch Java API to achieve different statistical functions, and show the graphical results with ECharts component. [Results] The real-time statistical analysis system of DSpace logs can get the browse rankings of items, collections and communities, get the download rankings of bitstreams, and get the regional rankings of website access, and so on. The statistics time can be customized by user, and the statistical result can be showed in different forms. [Conclusions] Using Logstash and ElasticSearch to achieve statistical analysis of DSpace logs has many excellences, just like no need to modify the code of DSpace, simple installation and deployment of the components, man-machine interactive query, fast and real-time, and rich forms to show the results.