面向Storm的自适应粒度监控组件设计与实现

Design and implementation of adaptive granularitymonitoring component in Storm

针对Storm系统提供的监控信息不全面且无法直接获取、监控粒度固定的问题,设计和实现一个自适应粒度的可视化监控组件。采用自适应粒度监控算法提供不同粒度的控制数据,结合网络文件系统、数据库和腾讯云图技术,从Storm UI和日志文件中获取、存储和统计拓扑运行状态信息,以图表的形式在网页中动态地显示监控数据。测试结果表明,监控组件能够准确地追踪集群和拓扑状态变化,从多拓扑和单拓扑的角度可视化监控数据,提高监控的灵活性、多样性和直观性,支持系统资源管理和调度策略调试。

Aiming at the problems that the monitoring information provided by storm system is not comprehensive and cannot be obtained directly,and the monitoring granularity is fixed,an adaptive granularity visual monitoring component was designed and implemented.Adaptive granularity monitoring algorithm was adopted to provide control data with different granularity,and network file system,database and Tencent cloud image technology were combined to obtain,store and count topology operation status information from storm UI and log files,and monitoring data were dynamically displayed in the form of charts in web pages.The test results show that the monitoring component can accurately track state changes of clusters and topologies,visualize the monitoring data from the perspective of multi topology and single topology,improve the flexibility,diversity and intuition of monitoring,and support system resource management and scheduling strategy debugging.