摘要
海量气象数据的长期归档保存是气象局的重要业务之一,传统的数据归档方式已经不能满足需要。基于多线程流水线的气象数据光盘自动归档方法可以解决海量数据归档的问题。利用该方法针对这一问题提出很好的解决方案,并完成了系统的开发、获得较好应用效果。系统通过基于线程流水线技术的多任务并行处理机制,大大缩短了海量数据的分类、压缩、光盘数据生成与刻录的整体运行时间。该系统每天能处理981.8GB,刻录数据577.3GB(以50G的蓝光光盘为例)。
Long term archival storage of mass meteorological data is one of the important businesses of the meteorological bureau. The traditional way of data archiving has been unable to meet the needs.The meteorological data compact disc auto archiving system gives out a perfect solution for this problem, which has completed system development and gained good application results. This system integrates data processing, disc burning, cover printing, and disc management, which solved the long term storage, auto archiving, and management problems of massive meteorological data. The whole processing time of massive data classifying, compressing, disc data organizing and burning is drastically reduced by using the multitask parallel mechanism based on muhithreading pipeline technology. The system can handle 981.8GB data and record 577.3GB data per day
作者
张金标
张恩红
Zhang Jinbiao;Zhang Enhong(Guangdong Meteorological Survey Data Center, Guangzhou Guangdong 510641)
出处
《气象研究与应用》
2018年第2期94-97,100,共5页
Journal of Meteorological Research and Application
关键词
海量数据
多线程
流水线
并行处理
自动归档
massive data
multi-thread
pipeline
（take the 50G blue light disc as an example） parallel processing
auto archiving
