发布订阅模式中间件中高效事件过滤器的研究

Highly Efficient Event Filter in the Publish/Subscribe Middleware

大规模分布式计算环境具有动态多变,结构复杂等一系列特点,为了满足该环境下高效地分发大批量数据给感兴趣的信息消费者,因此需要设计一种基于发布/订阅模式的中间件,使得高带宽的数据流能够被快速地过滤和转发。中间件把复杂事件流解析后,输入到一系列的轻量级组件形成的流水线中过滤,大幅提高了系统的效率.对核心模块事件过滤器,设计了高效的匹配算法,采用多级索引来加速查询,位运算压缩空间,对订阅做聚类,减少了重复冗余的匹配,有效地提高了系统的吞吐量,并且使得整个系统具有异步、松散耦合、多对多通信等特点,以及良好的可扩展性,能够适用于大规模的分布式计算环境.

Large scale distributed computing environment with dynamic,complex structure and a series of characteristics,to satisfy the need of distributing massive data to insterested message consumer in this enviroment,we need to design a publish/subscrible based middleware.The complex event message was broken-up into a pipeline,which is consisted of many light-weight filtering components,improved the efficiency of the system.In addition,develop a highly efficient algorithm for the core module event filter.By using multi-dimensional indexing,bitwise operators,do clustering for the subscription,reduce the redundant matching.Improve the system throughput,make it have asynchronous,many-to-many and loosely-coupled communication properties,good scalability,and can be used in the large scale of distributed computing environment.