Permuting Web Graphs

  • Paolo Boldi
  • Massimo Santini
  • Sebastiano Vigna
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5427)

Abstract

Since the first investigations on web graph compression, it has been clear that the ordering of the nodes of the graph has a fundamental influence on the compression rate (usually expressed as the number of bits per link). The author of the LINK database [1], for instance, investigated three different approaches: an extrinsic ordering (URL ordering) and two intrinsic (or coordinate-free) orderings based on the rows of the adjacency matrix (lexicographic and Gray code); they concluded that URL ordering has many advantages in spite of a small penalty in compression. In this paper we approach this issue in a more systematic way, testing some old orderings and proposing some new ones. Our experiments are made in the WebGraph framework [2], and show that the compression technique and the structure of the graph can produce significantly different results. In particular, we show that for the transpose web graph URL ordering is significantly less effective, and that some new orderings combining host information and Gray/lexicographic orderings outperform all previous methods. In particular, in some large transposed graphs they yield the quite incredible compression rate of 1 bit per link.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Randall, K., Stata, R., Wickremesinghe, R., Wiener, J.L.: The LINK database: Fast access to graphs of the Web. Research Report 175, Compaq Systems Research Center, Palo Alto, CA (2001)Google Scholar
  2. 2.
    Boldi, P., Vigna, S.: The WebGraph framework I: Compression techniques. In: Proc. of the Thirteenth International World Wide Web Conference, Manhattan, USA, pp. 595–601. ACM Press, New York (2004)Google Scholar
  3. 3.
    Kumar, R., Raghavan, P., Rajagopalan, S., Sivakumar, D., Tompkins, A., Upfal, E.: The Web as a graph. In: PODS 2000: Proceedings of the nineteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pp. 1–10. ACM Press, New York (2000)Google Scholar
  4. 4.
    Bharat, K., Broder, A., Henzinger, M., Kumar, P., Venkatasubramanian, S.: The Connectivity Server: fast access to linkage information on the Web. Computer Networks and ISDN Systems 30(1-7), 469–477 (1998)CrossRefGoogle Scholar
  5. 5.
    Blandford, D.K., Blelloch, G.E.: Index compression through document reordering. In: Data Compression Conference, pp. 342–351. IEEE Computer Society, Los Alamitos (2002)Google Scholar
  6. 6.
    Shieh, W.Y., Chen, T.F., Shann, J.J.J., Chung, C.P.: Inverted file compression through document identifier reassignment. Inf. Process. Manage 39(1), 117–131 (2003)CrossRefMATHGoogle Scholar
  7. 7.
    Silvestri, F.: Sorting out the document identifier assignment problem. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECiR 2007. LNCS, vol. 4425, pp. 101–112. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Blanco, R., Barreiro, Á.: Document identifier reassignment through dimensionality reduction. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 375–387. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Knuth, D.E.: The Art of Computer Programming. In: Fascicle 2: Generating All Tuples and Permutations (Art of Computer Programming), vol. 4. Addison-Wesley Professional, Reading (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Paolo Boldi
    • 1
  • Massimo Santini
    • 1
  • Sebastiano Vigna
    • 1
  1. 1.Dipartimento di Scienze dell’InformazioneUniversità degli Studi di MilanoItaly

Personalised recommendations