用MapReduce实现天文星表交叉认证

MapReduce for astronomical cross-matching

天文星表的交叉认证是天文研究中非常重要的基础工作。新巡天项目和更强大望远镜的投入使用,使天文数据爆炸增长,数据量的增加使得两个星表之间的交叉认证变得非常耗时。描述了如何利用MapReduce实现并行天文星表交叉认证,综合考虑了算法与体系结构的匹配问题,并给出了在大数据天文星表交叉认证工作的性能评估,通过与广泛使用的PostgreSQL数据库的比较,证明了基于MapReduce交叉认证方法的有效性。

As a basic and indispensable step,the astronomical cross-match is facing a data avalanche. With the completion of new sky survey projects and powerful telescopes,current cross-matching methods cannot be performed on demand for large scale astronomical data sets. This paper introduced MapReduce framework to solve this problem. It carefully considered the mapping of cross-matching algorithm on map and reduce phases. Performance evaluation shows that the MapReduce-based cross-matching can outperform the traditional one on PostgreSQL. As the knowledge,it is the first effort to adopt MapReduce for astronomical cross-matching problem.