期刊文献+

一个精细粒度实时计算资源管理系统 被引量:18

A Fine-grained,Real Time HPC Resource Management System
下载PDF
导出
摘要 由于相应业务系统软件的缺乏,国家级气象高性能计算机的资源管理措施落后于能力建设的发展。对此,该文提出了一个精细粒度实时计算资源管理系统。系统设计紧密围绕着目前竞争最为激烈的计算资源,采用资源虚拟单元GCU作为资源使用的计量单位,屏蔽了不同高性能计算机系统的体系结构差异,实现了计算资源细粒度的统一量化统计。系统可分为用户接口层、资源管理层、HPC系统层等3个层次,根据与网格平台软件不同结合方式以两种方式运行。在国家气象信息中心完成了系统的研发、部署和试验运行,根据试验运行的部分数据进行了用户单位和用户个人的计算资源使用的统计分析。目前,计算资源管理系统成果已成功应用到国家级气象高性能计算机计算资源的业务管理工作中。 In contrast to the rapid development of capability construction, resource management of national meteorological high performance computers is left behind. Absence of operational software in resource management keeps system administrators from having a detailed knowledge of what's going on in national meteorological high performance computers and exerting effective control over resource allocations. Regarding existing problems, a fine grained, real time high performance computer resource management system is proposed. The system is designed to be a real time, fine grained one with cross-cluster (Grid) support. The system works closely with CPU hours resources under keen competition. With the introduction of GCU (General Computing Unit), a resource virtualization unit, to measure computing resources, diversities of computing resources in different high performance computer systems are shielded and fine grained uniform quantitative management is enabled by the system. The target users of the system include resource users, leaders of user organizations, resource system administrators, decision-makers etc. The system comprises three layers, namely, user interfaces, resource management, and high performance computer systems. Resource management layer, the primary layer, can be divided into resource accounting and allocation manager, Grid platform, and resource information database. With open source software from supercomputing centers abroad, Grid project funded by MOST, and RDBMS employed, the system has seen an implementation, deployment and experimental running in National Meteorological Information Center. Fundamental functions of resource accounting and allocation management have been implemented, including cluster system job accounting, resource accounts management, management, allocation and query of user and organizations, providing command line interface for users. PostgreSQL database technology is adopted as the resource information database, on which accounts, users, organizations, computer systems, job records, accounting and allocation relation tables are created. The software system has been deployed into the three partitions of IBM high performance computer system, Sunway 32I cluster, Sunway 32P cluster, IBM SP system, working with LoadLeveler, PBS. Information of users on national meteorological high performance computer systems have been sorted and updated, resulting in uniform UID and GID, and inserted into databases. Two layers of management, organizations (projects) and individuals, are established. Computing resources are evenly allocated to user organizations according to 200 per cent of the total available resource in terms of GCUs. Only resources allocated to their department can be used by individual users. The validity of resources are set to a season. Overdraft is allowed. Based on partial data collected during experimental run, initial statistical analyses are made to probe resource usage by user organizations and individuals. At present, the high performance computer resource system has been put into operational run and successfully applied to operation management.
出处 《应用气象学报》 CSCD 北大核心 2008年第4期507-512,共6页 Journal of Applied Meteorological Science
基金 科技部基础条件平台计划"国家气象网络计算应用系统建设"项目(2005DKA64005) 中国气象局气象新技术推广项目(CMATG2008M07)共同资助
关键词 国家级气象高性能计算机资源 资源管理 GCU 实时 精细粒度 national meteorological high performance computer resources resource management GCU(General Computing Unit) real time fine-grained,
  • 引文网络
  • 相关文献

参考文献16

二级参考文献48

  • 1[1]I Foster, C Kesselman. The Grid: Blueprint for a Future Computing Infrastructure. Morgan Kaufmann Publishers, 1999
  • 2[2]NASA Information Power Grid. http://www.nas.nasa.gov/IPG
  • 3[3]G Laszewski, I Foster. Usage of LDAP in Globus. http://www.globus.org
  • 4[4]Timothy A Howes, Mark C Smith. LDAP: Programming Directory-Enabled Application with Lightweight Directory Access Protocol. Macmillan Technical Publishing, 1997
  • 5[1]I Foster, C Kesselman. The Grid: Blueprint for a Future Computing Infrastructure. San Francisco, California: Morgan Kaufmann Publishers, 1999
  • 6[2]Basney, M Livny. Deploying a high throughput computing cluster. In: Rajkumar Buyya ed. High Performance Cluster Computing, vol 1. Upper Saddle River, New Jersey: Prentice Hall Inc, 1999
  • 7[3]F Karpovich. Support for object placement in wide area heterogeneous distributed systems. University of Virginia Department of Computer, Tech Rep: CS-96-03, 1996
  • 8[4]Moore, G Fagg, A Geist et al. Scalable networked information processing environment. In: Proc of Supercomputing'97. San Jose, CA, 1997
  • 9[5]F Berman, R Wolski, S Figueira et al. Application level scheduling on distributed heterogeneous networks. In: Proc of Supercomputing 1996. Pittsburgh, PA, USA, 1996
  • 10[6]Vahdat, P Eastham, C Yoshikawa et al. WebOS: Operating system services for wide area applications. In: Proc of the 7th IEEE Int'l Symp on High Performance Distributed Computing. Chicago, IL, USA, 1998

共引文献118

同被引文献223

引证文献18

二级引证文献82

;
使用帮助 返回顶部