摘要
为简化江门中微子实验的离线数据处理流程,减少资源消耗,提出一种在分布式计算环境中进行数据处理的通用软件系统。基于信息传递接口实现节点间的通信与数据交换,使用Master/Worker架构对计算作业生命周期进行管理,包括计算作业拆分、计算资源分配以及计算任务执行与监控。测试结果表明,该系统具有良好的可扩展性,其产生的数据与人工逐步执行作业脚本运行模拟软件产生的数据一致。
In order to simplify the off-line data processing flow of Jiangmen Underground Neutrino Observation(JUNO) and reduce resource consumption,a general software system is proposed to process data in distributed computing environment.Based on Message Passing Interface(MPI),communication and data exchange between nodes are realized.Master/Worker architecture is used to manage the life cycle of computing jobs,including computing job splitting,computing resource allocation,computing task execution and monitoring.Test results show that the proposed system has good scalability,and the data generated by the system is consistent with the data generated by manual step-by-step execution of job scripts to run simulation software.
作者
赵宝琦
李卫东
邹佳恒
林韬
颜田
ZHAO Baoqi;LI Weidong;ZOU Jiaheng;LIN Tao;YAN Tian(School of Computer and Control Engineering,University of Chinese Academy of Sciences,Beijing 100049,China;Institute of High Energy Physics,Chinese Academy of Sciences,Beijing 100049,China)
出处
《计算机工程》
CAS
CSCD
北大核心
2019年第7期20-25,共6页
Computer Engineering
基金
中国科学院战略性先导科技专项(A类)(XDA10010900)
大科学装置联合基金(U1532258)
关键词
高能物理
江门中微子实验
离线数据处理
分布式计算
信息传递接口
high energy physics
Jiangmen Underground Neutrino Observation(JUNO)
off-line data processing
distributed computing
Message Passing Interface(MPI)
