一站式遥感大数据分布式管理与训练云平台

Distributed Deep Learning Platform for Massive Remote Sensing Data

随着遥感数据体量的不断增加,海量遥感数据管理与服务系统已经成为必不可少的基础设施。同时,业务功能的快速增长对遥感数据管理与服务平台的建设带来了新挑战,传统的基于单体架构的遥感数据管理与服务应用已经无法满足人们的服务需求。为此,本文设计并实现了一站式遥感大数据分布式管理与模型训练云平台,平台基于分布式存储-计算模型为基础的微服务架构,该架构能够保证平台的高可用性和易扩展性,为自研的遥感深度学习在线训练服务提供基础。经过测试,本平台遥感数据处理效率相较于传统框架都有明显提升,加速比大于2,满足当前遥感数据快速增长对数据管理和应用的需求。

With the continuous increase in the volume of remote sensing data,remote sensing data management and service platform has become an indispensable infrastructure.At the same time,functional requirements have evolved rapidly,which brings new challenges to the construction of remote sensing data management and service platform.Traditional applications based on monolithic architecture can no longer meet people’s needs for high availability and expandability.A onestop distributed management and deep learning platform for massive remote sensing data was designed and implemented based on the above background.Microservice architecture based on distributed storage-computing model was used.This architecture ensures high availability and expandability,providing the basis for self-developed online remote sensing deep learning services.After testing,this platform’s remote sensing data processing efficiency has been significantly improved compared with the traditional framework.The speedup ratio is greater than 2,which meets the requirements of data management and application platform in the context of the rapid growth of remote sensing data.