大数据信息可视化实时推送系统设计研究

Research on Design of Real-Time Push System for Big Data Information Visualization

互联网技术进一步成熟后,应用到生产环境中,可以有效提高对数据观测的实时性。而对时效性数据的如实采集和记录,会产生大量有效和冗余的数据信息。基于此背景,针对产生数据的时效性强、数据量大、有效性不确认、冗余混杂的特点,提出一种以数据流为中心的大数据信息可视化实时推送系统的设计流程。通过需求调研,形成有特征性的分组分类的统计聚合数据。通过数据结构分析,对数据指标统一格式、统一精度,结合推送需求平坦设计数据存储表。通过架构设计,综合考虑存储、线程、信息、时间调度和模块耦合等相关方面。通过布署调试,进行功能测试、实地测试和可行性测试,以确保系统设计的完成。通过在某地区单位的实际项目应用,验证了该系统设计流程的有效性与合理性。

The further maturity of Internet technology,when applied to the production environment can be effective for the realtime improvement of data observation,while the truthful collection and recording of time-sensitive data will in turn generate a large amount of valid and redundant data information.Based on this background,a design process of a data flow-centered big data information visualization real-time push system is proposed for the specific of the generated data with high timeliness,large data volume,unconfirmed validity and redundant mix.Through demand research,statistical aggregated data with characteristic grouping and classification are formed.Through the analysis of data structure,the format and accuracy of data indicators are unified,and the data storage table is designed flat according to the push demand.Through the architecture design,the storage,threading,information,time scheduling and module coupling are considered.Through deployment debugging,functional testing,field testing and feasibility testing to ensure the completion of the system design.The effectiveness and rationality of the system design process are verified by a practical project in a local unit.