Level-Biased Statistics in the Hierarchical Structure of the Web

  • Guang Feng
  • Tie-Yan Liu
  • Xu-Dong Zhang
  • Wei-Ying Ma
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3918)


In the literature of web search and mining, researchers used to consider the World Wide Web as a flat network, in which each page as well as each hyperlink is treated identically. However, it is the common knowledge that the Web is organized with a natural hierarchical structure according to the URLs of pages. Exploring the hierarchical structure, we found several level-biased characteristics of the Web. First, the distribution of pages over levels has a spindle shape. Second, the average indegree in each level decreases sharply when the level goes down. Third, although the indegree distributions in deeper levels obey the same power law with the global indegree distribution, the top levels show a quite different statistical characteristic. We believe that these new discoveries might be essential to the Web, and by taking use of them, the current web search and mining technologies could be improved and thus better services to the web users could be provided.


Hierarchical Structure Mining Technology Link Structure Spindle Shape Virtual Page 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Brin, S., Page, L., Motwami, R., Winograd, T.: The PageRank citation ranking: bring order to the web. Technical report, Computer Science Department, Stanford University (1998)Google Scholar
  3. 3.
    Broder, A.Z., Kumar, S.R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., Wiener, J.: Graph structure in the web: experiments and models. In: Proc. of the 9th WWW Conference, pp. 309–320 (2000)Google Scholar
  4. 4.
    Chung, F., Handjani, S., Jungreis, D.: Generalizations of Polya’s urn problem. Annals of Combinatorics 7, 141–153 (2003)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Eiron, N., McCurley, K.: Link structure of hierarchical information networks. In: Proc. Third Workshop on Algorithms and Models for the Web-Graph (2004)Google Scholar
  6. 6.
    Feng, G., Liu, T.-Y., Zhang, X.-D., Qin., T., Gao, B., Ma, W.-Y.: Level-based link analysis. In: Zhang, Y., Tanaka, K., Yu, J.X., Wang, S., Li, M. (eds.) APWeb 2005. LNCS, vol. 3399, pp. 183–194. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Kleinberg, J.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–622 (1999)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Klemm, K., Eguiluz, V.M.: Highly clustered scale-free networks. Phys. Rev. E 65, 036123 (2002)Google Scholar
  9. 9.
    Laura, L., Leonardi, S., Caldarelli, G., Rios, P.D.L.: A multi-layer model for the web graph. In: 2nd International Workshop on Web Dynamics, Honolulu (2002)Google Scholar
  10. 10.
    Newman, M.E.J.: The structure and function of complex networks. SIAM Review 45, 167–256 (2003)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Pennock, D.M., Flake, G.W., Lawrence, S., Giles, C.L., Glover, E.J.: Winners don’t take all: Characterizing the competition for links on the Web. In: Proceedings of the National Academy of Sciences (2002)Google Scholar
  12. 12.
    Ravasz, E., Barabasi, A.-L.: Hierarchical organization in complex networks. Phys. Rev. E 67, 026112 (2003)Google Scholar
  13. 13.
    Simon, H.A.: The Sciences of the Artifical, 3rd edn. MIT Press, Cambridge (1981)Google Scholar
  14. 14.
    Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small world’ networks. Nature 393, 440–442 (1998)CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Guang Feng
    • 1
  • Tie-Yan Liu
    • 2
  • Xu-Dong Zhang
    • 1
  • Wei-Ying Ma
    • 2
  1. 1.Microsoft Research AsiaBeijingP.R. China
  2. 2.MSPLAB, Department of Electronic EngineeringTsinghua UniversityBeijingP.R. China

Personalised recommendations