基于Daisy的天文卫星数据处理软件系统设计与实现

The Design and Implementation of Data Processing Software System for Astronomical Satellite

天文卫星数据处理是开展空间天文学领域科学研究的重要内容,也是当前研究的热点。然而,在大数据背景下,由于需要进行大规模数据处理,传统的数据处理方式已经无法满足当前应用的需求。针对这一问题,本文提出了一种数据处理软件系统,该系统基于Daisy(Data Analysis Integrated Software System)研发,实现天文卫星数据处理流程的自动化和规范化。首先,通过分析卫星数据产品间的依赖关系,采用数据分块处理技术,实现系统内存管理;其次,通过对数据处理算法和工具进行标准化,实现底层代码的统一管理;最后,研发工作流调度平台,实现数据处理流程的自定义配置和实时监控。实验验证了该系统具有有效性和实用性,能够克服传统数据处理方法的不足,实现数据处理流程的自动化和规范化,为相关软件系统的设计提供指导。

Astronomical satellite data processing is one of the essential aspects of scientific research in the field of space astronomy,which is currently a hot research topic.However,because large-scale data processing is required,traditional methods to process data do not satisfy the requirements of current applications under the background of big data.To address this issue,this paper proposes a data processing software system that realizes the automation and standardization of data processing for astronomical satellite.First,through analyzing the relationship among different satellite data products,data chunking technique is adopted to achieve the memory management.Second,by standardizing data processing algorithms and tools,the unified management of underlying code is realized.Finally,a workflow scheduling platform is developed to achieve the custom configuration and real-time monitoring of the data processing workflow.Experiments were conducted to verify the availability and practicality of the proposed system,and the results show that it is can overcome the deficiencies in traditional methods.The system also achieves the automation and standardization of data processing for astronomical satellite,which can provide guidance for the design of related software systems.