一种灵活可扩展的跨洲际数据放置实验床

A Flexible and Scalable Intercontinental Testbed for Data Placement Research

随着大数据时代的到来,不同领域的科研人员往往需要跨科研机构甚至是跨国家地域的移动大规模科学数据,从而更好的共享科研数据并促成跨学科协同分析,加速观测数据向科学发现的转化。然而,现有的研究工作缺乏对广域网环境下影响数据放置性能、开销和资源消耗等因素的深入分析,这些因素包括数据集中文件的数量和大小分布、源和目的节点文件系统的性能特性、各节点间网络性能的波动以及IPv4/IPv6网络协议的可用性等。为了更好的研究广域网环境下影响科学数据放置性能的多种因素,本文在中美真实网络环境下搭建了一种灵活可扩展的跨洲际数据放置实验床,并将其作为基础设施开放给科研人员用于运行多种类型的数据放置实验。

The emerging cloud computing and the era of big data promote researchers from different discipline to share large amount of scientific data across institutes and geological boundaries, in order to establish international collaborations acceleratingthe transition fromobservations to scientific discoveries. However, existing work lacks the understanding of various factors thataffect the performance, cost and resourceconsumption of data placement in the wide area network environment. These factors include the number and size of the files in the dataset, the characteristic of the filesystem on the source and destination nodes, the fluctuation of the network performance between the nodes, as well as the availability of IPv4/IPv6 network protocol. To better understand the factors that affect the performance of scientific data placement in wide area networkenvironment, we propose a flexible and scalable intercontinental testbed between China and USusing the real network, which is open to the researchers as a facility to conduct various data placement experiments.