分布式实时存储环境下的FITS数据一致性研究

Research on the Consistency of FITS Data in the Process of Distributed Real-time Storage

在大型太阳望远镜观测数据的分布式高速实时存储过程中,作为元数据的FITS头和FITS数据文件的不一致,导致应用程序不能根据构建在元数据上的索引信息检索到相对应的FITS数据文件。研究了这种环境下,两类数据的一致性模型;分析总结了二者在实时存储时的一致性保障机制和解决方案,提出了应用改进的两段提交协议来协商这两种数据之间的一致性关系,并给出了两段提交协议在高速数据存储背景下的算法实现,研究过程和方法能够对其他科学领域的相关应用提供参考。

In the process of a distributed high-speed real-time storage of data from a large solar telescope, inconsistency occurs between FITS header, a kind of meta-data, and FITS data files, which causes applications unable to retrieve correct FITS data files when applying the index information built on the meta- data. In this paper, models of consistency between the above-mentioned two types of data are studied during high-speed data storage; and the mechanism and solutions are summarized to achieve consistency for real-time data storage. A two-phase commit protocol （2PC） is proposed to ensure consistency, and 2PC algorithm implementation in high-speed data storage is given. The present study on different applications of data storage, with its unique research process and methods, can serve as references to other fields of scientific study.