基于Spring Cloud的EAST集成数据访问系统设计与实现

Design and Implementation of EAST Integrated Data Access System Based on Spring Cloud

EAST实验运行期间,会产生超过1 PiB的庞大数据量,基于实验的实际需求和数据结构特点,建立了多个数据系统。随着数据规模和系统访问量的不断增长,现有系统在功能维护和扩展等方面逐渐暴露出局限性,同时缺乏数据状态的有效监测和统一的访问机制。基于微服务架构设计,构建了EAST集成数据访问系统。以EAST实验数据类型为基准,建立多个支持单独开发部署的数据微服务,通过Spring Cloud框架组件,实现微服务集成管理;以Kafka为消息中间件,构建数据引擎,完成实验数据状态信息的异步采集和处理;采用VUE技术,构建门户系统,实现数据服务的统一访问。该系统成功突破了单体架构的诸多限制,不仅实现了EAST实验数据的状态监测功能,还为用户提供了一个综合性的EAST数据服务访问平台,进一步确保了EAST数据系统的高可用性。

During the EAST experiment,a large amount of data exceeding 1 PiB will be generated.Based on the actual needs of the experiment and the characteristics of the data structure,multiple data systems have been established.With the continuous growth of data scale and system access,existing systems has gradually exposed its limitations in terms of functional maintenance and expansion,while lacking effective monitoring of data status and a unified access mechanism.Based on the microservice architecture design,an EAST integrated data access system was constructed.Based on the EAST experimental data type,establish multiple data microservices that support independent development and deployment,and implement microservice integrated management through the Spring Cloud framework components.Using Kafka as the message middleware,a data engine is built to complete asynchronous collection and processing of experimental data status information.Using VUE technology,a portal system is built and unified access to data services is achieved.The system has successfully overcome many limitations of the single architecture,not only achieving the status monitoring function of EAST experimental data,but also providing users with a comprehensive EAST data service access platform,further ensuring the high availability of the EAST data system.