基于内存的临时文件中转系统研究与设计

RESEARCH AND DESIGN OF TEMPORARY FILE TRANSFER SYSTEM BASED ON MEMORY

由于超快的CPU内核与相对慢的存储器之间的速度差距在日渐增大,存储器系统可能成为当前制约系统性能的主要瓶颈。"天河二号"等超级计算机的问世,为需要大规模计算的软件系统提供了强大的计算能力支撑。但是对于那些会产生大量的临时小文件的系统,天河的集中式存储会严重影响到整体性能。为了解决上述问题,提出利用Linux消息队列、Socket网络编程、文件I/O等技术研究实现一个基于内存的临时文件中转系统。并且在"天河二号"超级计算机的漏洞挖掘系统上进行了应用对比测试,结果良好。这对改善"天河二号"超级计算机集中式存储的瓶颈具有指导意义。

Due to the increasing speed gap between an ultra-fast CPU core and a relatively slow memory,memory systems can be the major bottleneck constraining system performance.The advent of″Tianhe-2″and other supercomputers has provided powerful computing power support for software systems that need large-scale computing.But for those systems that generate a lot of temporary files,the Tianhe centralized storage will seriously affect the overall performance.In order to solve the above problems,this paper used Linux message queue,Socket network programming,file I/O technology to achieve a temporary memory-based file transfer system.We conducted a comparative test on the″Tianhe-2″supercomputer vulnerability mining system.The result of study is good,which has a guiding significance for improving the bottleneck of centralized storage of“Tihanhe-2”.