Data archiving is one of the most critical issues for modern astronomical observations.With the development of a new generation of radio telescopes,the transfer and archiving of massive remote data have become urgent ...Data archiving is one of the most critical issues for modern astronomical observations.With the development of a new generation of radio telescopes,the transfer and archiving of massive remote data have become urgent problems to be solved.Herein,we present a practical and robust file-level flow-control approach,called the Unlimited Sliding-Window(USW),by referring to the classic flow-control method in the TCP protocol.Based on the USW and the Next Generation Archive System(NGAS)developed for the Murchison Widefield Array telescope,we further implemented an enhanced archive system(ENGAS)using ZeroMQ middleware.The ENGAS substantially improves the transfer performance and ensures the integrity of transferred files.In the tests,the ENGAS is approximately three to twelve times faster than the NGAS and can fully utilize the bandwidth of network links.Thus,for archiving radio observation data,the ENGAS reduces the communication time,improves the bandwidth utilization,and solves the remote synchronous archiving of data from observatories such as Mingantu spectral radioheliograph.It also provides a better reference for the future construction of the Square Kilometer Array(SKA)Science Regional Center.展开更多
The construction of demonstration digital archives is a key project for the innovative development of the archival industry vigorously promoted by the National Archives Administration. Taking the construction of the S...The construction of demonstration digital archives is a key project for the innovative development of the archival industry vigorously promoted by the National Archives Administration. Taking the construction of the Smart Archives in Dongcheng District, Beijing as an example, this paper deeply explores the work objectives, functional requirements, and technical means of creating a demonstration digital archive. One is scientific planning, which is building the basic framework of digital archives. Proposed the infrastructure of “three major supports, five major platforms, and three major guarantees”. The second is to highlight key points and improve the basic functions of digital archives. Including hardware construction, software configuration, digitization of library archives, electronic archive management, sharing and utilization of digital archive information, and formulation of management systems for digital archives. The third is to strive for practical results and deepen the construction of smart libraries. This includes vigorously promoting the construction of smart archive centers, establishing a comprehensive management platform for smart libraries, and introducing new-generation information technology to achieve the integration of smart libraries in order to explore some mature experiences and methods that can be referenced for the creation of demonstration digital archives.展开更多
基金supported by the National Key Research and Development Program of China(2020SKA0110300)the Joint Research Fund in Astronomy(U1831204 and U1931141)under cooperative agreement between the National Natural Science Foundation of China(NSFC)+7 种基金the Chinese Academy of Sciences(CAS)(NSFC,No.11903009)the Funds for International Cooperation and Exchange of the NSFC(11961141001)Yunnan Key Research and Development Program(2018IA054)The Key Science and Technology Program of Henan Province(Nos.202102210152,212102210611 and 202102210125)the Research and Cultivation Fund Project of Anyang Normal University(AYNUKPY-2019-24 and AYNUKPY-2020-25)supported by Astronomical Big Data Joint Research Centerco-founded by the National Astronomical ObservatoriesChinese Academy of Sciences and Alibaba Cloud。
文摘Data archiving is one of the most critical issues for modern astronomical observations.With the development of a new generation of radio telescopes,the transfer and archiving of massive remote data have become urgent problems to be solved.Herein,we present a practical and robust file-level flow-control approach,called the Unlimited Sliding-Window(USW),by referring to the classic flow-control method in the TCP protocol.Based on the USW and the Next Generation Archive System(NGAS)developed for the Murchison Widefield Array telescope,we further implemented an enhanced archive system(ENGAS)using ZeroMQ middleware.The ENGAS substantially improves the transfer performance and ensures the integrity of transferred files.In the tests,the ENGAS is approximately three to twelve times faster than the NGAS and can fully utilize the bandwidth of network links.Thus,for archiving radio observation data,the ENGAS reduces the communication time,improves the bandwidth utilization,and solves the remote synchronous archiving of data from observatories such as Mingantu spectral radioheliograph.It also provides a better reference for the future construction of the Square Kilometer Array(SKA)Science Regional Center.
文摘The construction of demonstration digital archives is a key project for the innovative development of the archival industry vigorously promoted by the National Archives Administration. Taking the construction of the Smart Archives in Dongcheng District, Beijing as an example, this paper deeply explores the work objectives, functional requirements, and technical means of creating a demonstration digital archive. One is scientific planning, which is building the basic framework of digital archives. Proposed the infrastructure of “three major supports, five major platforms, and three major guarantees”. The second is to highlight key points and improve the basic functions of digital archives. Including hardware construction, software configuration, digitization of library archives, electronic archive management, sharing and utilization of digital archive information, and formulation of management systems for digital archives. The third is to strive for practical results and deepen the construction of smart libraries. This includes vigorously promoting the construction of smart archive centers, establishing a comprehensive management platform for smart libraries, and introducing new-generation information technology to achieve the integration of smart libraries in order to explore some mature experiences and methods that can be referenced for the creation of demonstration digital archives.