摘要
数据流成为日益重要的数据密集型应用.离线分析处理是对数据流产生的海量日志数据进行随意的统计查询,单个查询处理的数据量在上百GB,及时的响应时间和扩展性对传统数据库提出巨大挑战.本文以网络监控为背景,分析了离线分析处理的应用特征,提出了一种无共享的并行查询中间件,利用多策略及DBMS实现局部结果的汇总,通过具体的执行过程,分析了不同类型查询的扩展性.
Data stream is becoming an important data-intensive application. Off-line analysis process optionally querys massive logged data that data stream produces, and the single querying may process hundreds of GB data. Its timeliness and scalability present great challenge to traditional database. In this paper, the applied characteristics of data stream off-line analysis are analysed based on network monitoring management, a shared-nothing parallel database middleware is presented,and the summarizing of local result is realized by using DBMS and multi-policy. Finally, the scalabilities of different kinds＇of query are analysed by specific execution process.
出处
《兰州交通大学学报》
CAS
2008年第4期102-105,共4页
Journal of Lanzhou Jiaotong University
关键词
数据流离线分析
无共享
并行查询中间件
可扩展性
data stream off-line analysis
shared-nothing
parallel query middleware
scalabilities
