基于NoSQL的FITS文件头元数据存储和查询研究

Research on metadata storage and querying of FITS file based on NoSQL

随着大型天文望远镜的投入使用,观测台站正面临PB量级的海量数据存储、快速检索难题;同时由于在数据检索中起着关键作用的FITS文件头的可变性,导致难以使用传统的关系型数据库来建立可适应这种变化需求的非结构化数据模型。针对这个难题,提出了使用NoSQL对天文上广泛使用的FITS文件头中所包含的可变元数据信息进行存储和查询;讨论了关系型数据模型存储可变FITS文件头的不足;分析了NoSQL存储可变FITS头元数据信息的可行性;使用形式化的关系型代数对这种存储查询方式进行了一般化的讨论。通过具体查询实例验证了该方案在存储天文可变FITS文件头的有效性和可行性。

With a large telescope in use, observing stations are facing problems of PB-level data storage and data retrieval. Because of the variability of the FITS file header, it is difficult to utilize the traditional relational database to store and retrieve the unstructured data efficiently. To address these problems, this paper proposed using NoSQL to store and query the metadata of the FITS file header. It discussed the shortcomings of using relational data model to store the variable header of a FITS file. It described the process of data storage and query, analyzed the feasibility of using NoSQL to store the variable FITS header, and further employed the method of relational algebra to express the process formally. A specific query instance proves the effectiveness and feasibility of using NoSQL to store the variable astronomical FITS header.