期刊文献+

基于强连通分量的个性化的网页排名高效算法 被引量:3

Strongly Connected Components Based Efficient PPR Algorithms
下载PDF
导出
摘要 个性化的网页排名(PPR)是一种常用的图结点排名方法.随着图的规模变得越来越大,如何快速地计算出PPR逐渐成为大家研究的关注热点.该文的最终目的即是为了提高PPR的计算效率.现有的各种优化算法可大体分为分布式算法和串行算法,其主要思路均是通过将大图上的计算分割到多个小子图上进行计算,但不同分块间的数据通信量往往很大而且通信次数频繁.该文提出的基于强连通分量的算法可有效解决此类问题.其主要计算过程为,首先快速将大量与计算无关的结点和边剪切掉,其次通过某种策略将在大图上的计算转化到多个强连通分量子图上计算,使得各分量子图之间的数据传递只需一次即可完成.该文基于强连通分量算法,不仅减少了分布式算法子图间的通信量,而且降低了串行算法的磁盘读写I/O频率,同时还保证了算法的准确度几乎不受损失.实验结果表明该文提出的算法可显著提高PPR的计算效率. Personalized PageRank (PPR) is usually employed to rank the nodes of graphs. Due to the ever increasing volume of graph, how to improve the efficiency of PPR computations has become a research focus. Thus the purpose of this paper is to improve the efficiency of PPR computations. The existing optimization algorithms can he generally classified into two categories: distributed algorithm and serial algorithm, and the general approach of them is mainly through partitioning the computations on the big graph into computations on multiple smaller sub-graphs, but the communication between sub-graphs usually involve a large amount of data and is of a high frequency. The SCC (Strongly Connected Component) based algorithms proposed in this paper can resolve these problems effectively. The main computation steps of them is: first identify and remove volumes of unrelated nodes and edges quickly before PPR computation, then transform the PPR computations on big graph into that on multiple SCC sub-graphs, which make the multiple data communications between sub-graphs turn into one time communication. The SCC based algorithm in this paper can reduce not only the communication amount between SCCs but also the storage I/O frequency, while keeping high algorithm accuracy. The experiments demonstrate that the algorithms proposed in this paper can make obvious improvements for the PPR computation efficiency.
出处 《计算机学报》 EI CSCD 北大核心 2017年第3期584-600,共17页 Chinese Journal of Computers
基金 国家"九七三"重点基础研究发展规划项目基金(2012CB316201) 国家自然科学基金面上项目(61472070)资助~~
关键词 个性化的网页排名 分布式算法 串行算法 强连通子图 通信量 I/O频率 personalized page-rank distributed computing serial computing strongly connectedcomponent communication amount I/O frequency
  • 相关文献

参考文献1

二级参考文献33

  • 1Dean J, Ghemawat S. MapReduce: Simplified dala processing on large clusters//Proceedings of the Conference on Operating System Design and Implementation(OSDU04,). San Francisco, USA, 2004: 137-150.
  • 2Thusoo A, Sarma J S, JainN, Shao Z, Chakka P, Anthony S, Liu H, Wyckoff P, Murthy R. Hive: A warehousing solution over a map-reduce framework//Proceedings of the Conference on Very Large Databases (VLDB' 09). Lyon, France, 2009:1626-1629.
  • 3Olston C, Reed B, Srivastava U, Kumar R, Tomkins A. Pig Latin: A not-so-foreign language for data processing//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data (SIGMOD' 08). Vancouver, BC, Canada, 2008:1099 1110.
  • 4Bu Y, Howe B, Balazinska M, Ernst M D. HaLoop.. Efficient iterative data processing on large clusters//Proceedings of the Conference on Very Large Databases (VLDB' 10). Sin gapore, 2010:285-296.
  • 5Ekanayake J, Li H, Zhang B, Gunarathne T, Bae S-H, Qiu J, Fox G. Twister: A runtime for iterative MapReduce// Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing. Chicago, Illinois, USA, 2010:810-818.
  • 6Wilson G V. Practical Parallel Programming. Cambridge, MA.. MIT Press, 1995.
  • 7Valiant L G. A bridging model for parallel computation. Communications of the ACM, 1990, 33(8): 103-111.
  • 8Dean J, Ghemawat S. MapReduce: A flexible data processing tool. Communications of the ACM, 2010, 53(1): 72-77.
  • 9Pavlo A, Paulson E, Rasin A, Abadi D J, DeWitt D J, Mad den S, Stonebraker M. A comparison of approaches to large scale data//Proceedings of the 2009 ACM SIGMOD Interna tional Conference on Management of Data (SIGMOD' 09) New York, USA, 2009:165-178.
  • 10Stonebraker M, Abadi D J, DeWitt D J, Madden S, Paulson E, Pavlo A, Rasin A. MapReduce and parallel DBMSs: Friends or foes? Communications of the ACM, 2010, 53(1) : 64-71.

共引文献44

同被引文献17

引证文献3

二级引证文献6

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部