This paper describes the architecture of global distributed storage system for data grid. It focue on the management and the capability for the maximum users and maximum resources on the Internet, as well as performan...This paper describes the architecture of global distributed storage system for data grid. It focue on the management and the capability for the maximum users and maximum resources on the Internet, as well as performance and other issues.展开更多
When using distributed storage systems to store gridded remote sensing data in large,distributed clusters,most solutions utilize big table index storage strategies.However,in practice,the performance of big table inde...When using distributed storage systems to store gridded remote sensing data in large,distributed clusters,most solutions utilize big table index storage strategies.However,in practice,the performance of big table index storage strategies degrades as scenarios become more complex,and the reasons for this phenomenon are analyzed in this paper.To improve the read and write performance of distributed gridded data storage,this paper proposes a storage strategy based on Ceph software.The strategy encapsulates remote sensing images in the form of objects through a metadata management strategy to achieve the spatiotemporal retrieval of gridded data,finding the cluster location of gridded data through hash-like calculations.The method can effectively achieve spatial operation support in the clustered database and at the same time enable fast random read and write of the gridded data.Random write and spatial query experiments proved the feasibility,effectiveness,and stability of this strategy.The experiments prove that the method has higher stability than,and that the average query time is 38%lower than that for,the large table index storage strategy,which greatly improves the storage and query efficiency of gridded images.展开更多
面向对象的存储系统在研究、工程以及服务领域均得到了广泛的应用.在面向对象的存储系统中,元数据的负载均衡对于提高整个系统的I/O性能具有重要的作用.现有的元数据负载均衡策略不能动态地平衡元数据的访问负载,而且自适应性以及容错...面向对象的存储系统在研究、工程以及服务领域均得到了广泛的应用.在面向对象的存储系统中,元数据的负载均衡对于提高整个系统的I/O性能具有重要的作用.现有的元数据负载均衡策略不能动态地平衡元数据的访问负载,而且自适应性以及容错特性有待提高.提出了一种自适应的分布式元数据负载均衡机制(adaptabledistributed load balancing of metadata,简称ADMLB),包含基本的负载均衡算法和分布式的增量负载均衡算法.采用基本的负载均衡算法按照服务器的性能公平地分布负载,使用分布式的负载均衡算法定时地调整负载的分布.ADMLB采取分布式的方法均衡地在元数据服务器之间分布负载,根据负载的变化自适应地进行调整,具有很好的容错特性,而且用户可以高效地定位元数据服务器.展开更多
文摘This paper describes the architecture of global distributed storage system for data grid. It focue on the management and the capability for the maximum users and maximum resources on the Internet, as well as performance and other issues.
基金This work was funded by the Department of Science and Technology of Henan Province through grant 201400210100the National Key R&D Program of China through grant 2019YFE0127000this work was supported by National Supercomputing Center in Zhengzhou.
文摘When using distributed storage systems to store gridded remote sensing data in large,distributed clusters,most solutions utilize big table index storage strategies.However,in practice,the performance of big table index storage strategies degrades as scenarios become more complex,and the reasons for this phenomenon are analyzed in this paper.To improve the read and write performance of distributed gridded data storage,this paper proposes a storage strategy based on Ceph software.The strategy encapsulates remote sensing images in the form of objects through a metadata management strategy to achieve the spatiotemporal retrieval of gridded data,finding the cluster location of gridded data through hash-like calculations.The method can effectively achieve spatial operation support in the clustered database and at the same time enable fast random read and write of the gridded data.Random write and spatial query experiments proved the feasibility,effectiveness,and stability of this strategy.The experiments prove that the method has higher stability than,and that the average query time is 38%lower than that for,the large table index storage strategy,which greatly improves the storage and query efficiency of gridded images.
文摘面向对象的存储系统在研究、工程以及服务领域均得到了广泛的应用.在面向对象的存储系统中,元数据的负载均衡对于提高整个系统的I/O性能具有重要的作用.现有的元数据负载均衡策略不能动态地平衡元数据的访问负载,而且自适应性以及容错特性有待提高.提出了一种自适应的分布式元数据负载均衡机制(adaptabledistributed load balancing of metadata,简称ADMLB),包含基本的负载均衡算法和分布式的增量负载均衡算法.采用基本的负载均衡算法按照服务器的性能公平地分布负载,使用分布式的负载均衡算法定时地调整负载的分布.ADMLB采取分布式的方法均衡地在元数据服务器之间分布负载,根据负载的变化自适应地进行调整,具有很好的容错特性,而且用户可以高效地定位元数据服务器.
