Hypergraph Partitioning for Faster Parallel PageRank Computation
The PageRank algorithm is used by search engines such as Google to order web pages. It uses an iterative numerical method to compute the maximal eigenvector of a transition matrix derived from the web’s hyperlink structure and a user-centred model of web-surfing behaviour. As the web has expanded and as demand for user-tailored web page ordering metrics has grown, scalable parallel computation of PageRank has become a focus of considerable research effort.
In this paper, we seek a scalable problem decomposition for parallel PageRank computation, through the use of state-of-the-art hypergraph-based partitioning schemes. These have not been previously applied in this context. We consider both one and two-dimensional hypergraph decomposition models. Exploiting the recent availability of the Parkway 2.1 parallel hypergraph partitioner, we present empirical results on a gigabit PC cluster for three publicly available web graphs. Our results show that hypergraph-based partitioning substantially reduces communication volume over conventional partitioning schemes (by up to three orders of magnitude), while still maintaining computational load balance. They also show a halving of the per-iteration runtime cost when compared to the most effective alternative approach used to date.
Unable to display preview. Download preview PDF.
- 1.Wolff, R.: Stochastic Modeling and the Theory of Queues. Prentice-Hall International Editions, Englewood Cliffs (1989)Google Scholar
- 2.Haveliwala, T.H.: Topic sensitive PageRank: A context-sensitive ranking algorithm for web search. Tech. Rep., Stanford University (March 2003)Google Scholar
- 7.Boman, E., Devine, K., Heaphy, R., Catalyurek, U., Bisseling, R.: Parallel hypergraph partitioning for scientific computing. Tech. Rep. SAND05–2796C, Sandia National Laboratories, Albuquerque, NM (April 2005)Google Scholar
- 9.de Jager, D.: PageRank: Three distributed algorithms. M.Sc. thesis, Department of Computing, Imperial College London, London SW7 2BZ, UK (September 2004)Google Scholar
- 11.Haveliwala, T.H., Kamvar, S.D.: The second eigenvalue of the google matrix. Tech. Rep., Computational Mathematics, Stanford University (March 2003)Google Scholar
- 12.Google (June 20, 2005), http://www.google.com/
- 13.Kamvar, S.D., Haveliwala, T.H., Manning, C.D., Golub, G.H.: Exploiting the block structure of the web for computing PageRank. In: Stanford database group tech. rep., Computational Mathematics, March 2003, Stanford University (2003)Google Scholar
- 14.Gleich, D., Zhukov, L., Berkhin, P.: Fast parallel PageRank: A linear system approach. Tech. Rep., Institute for Computation and Mathematical Engineering, Stanford University (2004)Google Scholar
- 15.Catalyurek, U.V., Aykanat, C.: A Fine-Grain Hypergraph Model for 2D Decomposition of Sparse Matrices. In: Proc. 8th International Workshop on Solving Irregularly Structured Problems in Parallel, San Francisco, USA (April 2001)Google Scholar
- 19.Trifunovic, A., Knottenbelt, W.: A Parallel Algorithm for Multilevel k-way Hypergraph Partitioning. In: Proc. 3rd International Symposium on Parallel and Distributed Computing, July 2004, pp. 114–121. University College Cork, Ireland (2004)Google Scholar
- 20.Davis, T.: University of Florida Sparse Matrix Collection (March 2005), http://www.cise.ufl.edu/research/sparse/matrices
- 21.UbiCrawler project, http://webgraph-data.dsi.unimi.it/