基于大数据的航空数据采集与处理系统研究与设计

Research and Design of the Data Acquisition and Processing System Based on Big Data

提供了一种基于大数据的航空作战数据采集与处理系统架构技术,包括:多源异构数据采集层,用于获取原始数据并发送给数据治理层;数据治理层,用于接收和对所述原始数据执行解析、清洗转换形成干净数据,并进行预处理形成主题数据;数据管理层,用于对数据生命周期、元数据、数据库、文件等进行管理及统计分析;数据服务层,用于提供数据智能缓存、事件分析、数据发布、数据挖掘等服务;数据应用层,用于接收所述主题数据进行任务复盘和回放,以及基于元数据进行数据地图、商业智能(BI:Business Intelligence)等有价值的数据分析应用。该方法能够有效提高航空数据管理的智能化水平,挖掘数据价值。

An architecture technology of data acquisition and processing system is provided for aerial operations based on large data,including:multi-source heterogeneous data acquisition layer for acquiring raw data and sending it to data processing layer;data processing layer for receiving and parsing the raw data,cleaning and transforming to form clean data.The data management layer is used to manage and analyze data life cycle,metadata,database,files,etc.The data service layer is used to provide data intelligent caching,event analysis,data publishing,data mining and other services.The data application layer is used to receive the said topics.Data are rewritten and replayed by tasks,and valuable data applications such as data map and BI analysis based on metadata.The method described can improve the intelligence level of aviation data management and mine the value of data.