利用ARC CE整合异构计算平台

Integrate Heterogeneous Computing Platforms with ARC CE

ARC(Advanced Resource Connector)CE(Computing Element)是Nordugrid开发的网格中间件中的对计算资源管理的组件。ARC CE相对于其他的对等计算资源网格组件,具有轻量级,易管理,认证简单,易扩展的特性,因此越来越广泛地被应用于各类网格应用平台上。高能物理实验ATLAS具有数据量大,计算量大的特点,采用了网格计算的方式整合了位于全球上百个计算中心的15万个CPU核进行协同计算。随着ATLAS Run2阶段开始,其所需计算资源急速增长,因此ATLAS在积极探索使用云计算资源,超算资源(HPC),志愿计算资源等。如何将这些异构的计算资源整合到ATLAS传统的网格计算平台,成为一个急需解决的重要问题。ARC CE由于所具有的易扩展和轻认证的属性,于是成为这一解决方案的核心组件。本文将以志愿计算为例,阐述如何利用ARC CE将动态的、不可信的志愿计算资源平台整合到现有的ATLAS网格计算环境,实现对用户和网格服务的双透明性。

ARC （Advanced Resource Connector） CE （Computing Element） is one ot the components and services of the grid middleware developed by Nordugrid. Its function is to manage the computing resources. Compared to the peering services of other grid middleware, ARC CE is lightweighted, easy to manage, with simple authentication, and scalable, thus it has been widely used to various grid computing infrastructures. The high energy physics experiment ATLAS generates massive volume of data, and requires huge amounts of computing resources, so a grid computing platform was built for ATLAS, which integrates computing resources from over a hundred computing centers which provides over 150K CPU cores to meet its computing requirements. Since the Run2 of ATLAS started in 2015, its computing requirements grows even bigger, so ATLAS also started to explore all sorts of opportunities of computing resources, such as cloud computing, HPC, and volunteer computing resources, to expand the availability of computing resources. How to harness and integrate all these heterogeneous computing resources into the ATLAS grid computing platform becomes one of the big challenges. ARC CE becomes the core component to solve this challenge due to its lightweight and simple authentication feature. In this paper, volunteer computing is used as an example to demonstrate how to harness and integrate the dynamical and un-trustable computing resources into the current ATLAS computing platform with the adoption and modification of ARC CE. This solution also provides transparency to both ATLAS end users and all ATLAS grid services.