云计算环境下科学工作流中的数据管理研究

A Research on Data Management for Science Workflow in Cloud

大数据时代,科学家必须使用多个科学工作流管理系统协同完成一项大型实验,来自不同环境和不同科学工作流管理系统的数据构成了科学大数据,科学大数据的产生为科学工作流管理系统中的数据管理带来挑战。科学工作流一般由若干个任务构成,这些任务对输入数据进行运算以产生新的数据为后续任务使用,这些数据需要暂存或者长期存储并在需要时候能够被检索。利用对象存储的优势,以两种不同的模式,对科学工作流的输入数据、中间数据和输出数据予以布局和优化存储,为云计算环境下科学工作流中的数据管理提供参考。

In the era of big data, scientists need to establish experimental model for different workflows and data structure. Data from different environments and different Scientific Workflow Management System forms big scientific data, which challenges to Scientific Workfiow Management System. Scientific workflow is usually composed by several tasks that operate the input data and generate new data for subsequent tasks. The data need to be store for late access. The model of perform scientific workflow in cloud environment is presented, and taking advantage of object-based storage, two schemes of data management for scientific workflow are provided, which help to layout and optimize saving all files include input data, intermediate data and output data. Some recommendations for data management for scientific workflow in cloud is proposed.