Python多核并行计算在海量星表交叉证认中的应用

The application of multi-core parallel computing using python language in cross-matching of massive catalogues

天文学研究中经常需要通过交叉证认将来自多波段多项目天文数据联系起来统一考虑.当前天文数据急剧增长,必然导致交叉证认的速度过慢.针对这一问题,提出一种在多核环境下使用Python语言进行高效并行计算的方法,与以往的研究结果相比,速度提高了若干倍.这为下一步的多波段数据统计研究和数据挖掘打下了良好的基础.

As astronomical data grows rapidly, cross-matching between huge catalogues that contain millions or billions of celestial objects becomes a hotspot of research. In this paper, we present a parallel cross-match program, which is written in Python language and is able to make full use of multi-core processors. We explain why Python programming language is chosen and how to implement parallel computing with it. A sky splitting function HTM is selected to partition catalogues. The results of experiments prove that our program has a significant performance advantage comparing with previous functions and lays a good foundation for further statistical research and data mining.