Exploring and Understanding Scientific Metrics in Citation Networks

  • Mikalai Krapivin
  • Maurizio Marchese
  • Fabio Casati
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 5)


This paper explores scientific metrics in citation networks in scientific communities, how they differ in ranking papers and authors, and why. In particular we focus on network effects in scientific metrics and explore their meaning and impact. We initially take as example three main metrics that we believe significant; the standard citation count, the more and more popular h-index, and a variation we propose of PageRank applied to papers (called PaperRank) that is appealing as it mirrors proven and successful algorithms for ranking web pages and captures relevant information present in the whole citation network. As part of analyzing them, we develop generally applicable techniques and metrics for qualitatively and quantitatively analyzing such network-based indexes that evaluate content and people, as well as for understanding the causes of their different behaviors. We put the techniques at work on a dataset of over 260K ACM papers, and discovered that the difference in ranking results is indeed very significant (even when restricting to citation-based indexes), with half of the top-ranked papers differing in a typical 20-element long search result page for papers on a given topic, and with the top researcher being ranked differently over half of the times in an average job posting with 100 applicants.


Scientific metrics Scientometric Page Rank Algorithm Paper Rank H-index Divergence metric in ranking results 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    de Solla Price, D.: Little Science - Big Science. Columbia Univ. Press, New York (1963)Google Scholar
  2. 2.
    Garfield, E.: Citation Indexing. ISI Press (1979)Google Scholar
  3. 3.
    Glänzel, W.: Bibliometrics as a research field, A course on theory and application of bibliometric indicators, Magyar Tudományos Akadémia, Course Handouts (2003)Google Scholar
  4. 4.
    Hirsch, J.E.: An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences 102(46), 16569–16572 (2005)CrossRefGoogle Scholar
  5. 5.
    Chen, P., Xie, H., Maslov, S., Redner, S.: Finding Scientific Gems with Google. Journal of Informetrics 1(1), 8–15 (2007)CrossRefGoogle Scholar
  6. 6.
    Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30(1-7), 107–117 (1998)CrossRefGoogle Scholar
  7. 7.
    Bianchini, M., Gori, M., Scarselli, F.: Inside PageRank. ACM Transactions on Internet Technology 5(1) (2005)Google Scholar
  8. 8.
    Del Corso, G.M., Gull, A., Romani, F.: Fast PageRank Computation via a Sparse Linear System. Internet Mathematics 2(3), 251–273 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Diligenti, M., Marco Gori, M., Maggini, M.: Web Page Scoring Systems for Horizontal and Vertical Search. In: WWW Conference 2002, USA, May 7-11 (2002)Google Scholar
  10. 10.
    Sun, Y., Giles, L.C.: Popularity Weighted Ranking for Academic Digital Libraries. In: 29th European Conference on Information Retrieval 2007, Rome, Italy, pp. 605–612 (2007)Google Scholar
  11. 11.
    Bernstam, E.V., Herskovic, J.R., Aphinyanaphongs, Y.: Using Citation Data to Improve Retrieval from MEDLINE. Journal of the American Medical Informatics Association 13(1), 96–105 (2006)CrossRefGoogle Scholar
  12. 12.
    Langville, A.N., Meyer, C.D.: Deeper Inside PageRank. Internet Mathematics 1(3), 335–380 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Kendall, M., Gibbons, J.D.: Rank Correlation Methods. Edward Arnold, London (1990)zbMATHGoogle Scholar

Copyright information

© ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering 2009

Authors and Affiliations

  • Mikalai Krapivin
    • 1
  • Maurizio Marchese
    • 1
  • Fabio Casati
    • 1
  1. 1.Department of Information Engineering and Computer ScienceUniversity of TrentoItaly

Personalised recommendations