Network Signatures of Success: Emulating Expert and Crowd Assessment in Science, Art, and Technology

  • Igor ZakhlebinEmail author
  • Emőke-Ágnes Horvát
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 689)


The success of scientific, artistic, and technological works is typically judged by human experts and the public. Recent empirical literature suggests that exceptionally creative works might have distinct patterns of citation. Given the recent availability of large citation and reference networks, we investigate how highly successful works differ from less successful ones in terms of a broad selection of centrality indices. Our experiments show that expert opinion is better emulated than popular judgment even with a single well-chosen index. Our findings further provide insights into otherwise implicit assumptions about indicators of success by evaluating the success of works based on the patterns of references that they receive.



Authors would like to thank Roberta Sinatra for valuable discussion and data on Nobel prize winners. Special thanks go to Noshir Contractor and SONIC lab members for supporting the preparation of this manuscript.I.Z. has been partially funded through the Russian academic excellence project “5–100”.


  1. 1.
    Bonabeau, E.: Decisions 2.0: the power of collective intelligence. MIT Sloan Manag. Rev. 45–52 (2009)Google Scholar
  2. 2.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  3. 3.
    Burton, R.E., Kebler, R.W.: The “half-life” of some scientific and technical literatures. Am. Doc. 11(1), 18–22 (1960). CrossRefGoogle Scholar
  4. 4.
    Chen, P., Xie, H., Maslov, S., Redner, S.: Finding scientific gems with google’s pagerank algorithm. J. Inf. 1(1), 8–15 (2007). CrossRefGoogle Scholar
  5. 5.
    Dreber, A., et al.: Using prediction markets to estimate the reproducibility of scientific research. Proc. Natl. Acad. Sci. 112(50), 15343–15347 (2015)CrossRefGoogle Scholar
  6. 6.
    Elberse, A.: Blockbusters: Hit-making, Risk-taking, and the Big Business of Entertainment. Henry Holt and Company, LLC (2013)Google Scholar
  7. 7.
    Fawcett, T.: An introduction to ROC analysis. Pattern Recognit. Lett. 27(8), 861–874 (2006). MathSciNetCrossRefGoogle Scholar
  8. 8.
    Garfield, E.: Citation indexes for science: a new dimension in documentation through association of ideas. Science 122(3159), 108–111 (1955). CrossRefGoogle Scholar
  9. 9.
    Garfield, E.: The history and meaning of the journal impact factor. JAMA 295(1), 90–93 (2006). CrossRefGoogle Scholar
  10. 10.
    Hahn, R., Tetlock, P.C.: Using information markets to improve public decision making. Harv. J. Law Public Policy 29(1), 213–289 (2005)Google Scholar
  11. 11.
    Hirsch, J.E.: An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. U. S. A. 102(46), 16569–16572 (2005). CrossRefzbMATHGoogle Scholar
  12. 12.
    Lehmann, S., Jackson, A.D., Lautrup, B.E.: Measures for measures. Nature 444(7122), 1003–1004 (2006). CrossRefGoogle Scholar
  13. 13.
    Louppe, G., Wehenkel, L., Sutera, A., Geurts, P.: Understanding variable importances in forests of randomized trees. In: Advances in Neural Information Processing Systems, pp. 431–439 (2013)Google Scholar
  14. 14.
    Monechi, B., Gravino, P., Servedio, V.D.P., Tria, F., Loreto, V.: Significance and popularity in music production. R. Soc. Open Sci. 4(7) (2017).
  15. 15.
    Mones, E., Pollner, P., Vicsek, T.: Universal hierarchical behavior of citation networks. J. Stat. Mech. Theory Exp. 2014(5), P05023 (2014).
  16. 16.
    Moore, D.A., et al.: Confidence calibration in a multiyear geopolitical forecasting competition. Manag. Sci. (2016).
  17. 17.
    Mukherjee, S., Romero, D.M., Jones, B., Uzzi, B.: The nearly universal link between the age of past knowledge and tomorrow’s breakthroughs in science and technology: the hotspot. Sci. Adv. 3(4), e1601315 (2017)Google Scholar
  18. 18.
    Newman, M.E.: Networks. An introduction. Oxford (2010)Google Scholar
  19. 19.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: Bringing order to the web. Tech. rep, Stanford InfoLab (1999)Google Scholar
  20. 20.
    Price, D.J.D.S.: Networks of scientific papers. Science pp. 510–515 (1965)Google Scholar
  21. 21.
    Reader, S.M., Morand-Ferron, J., Flynn, E.: Animal and human innovation: novel problems and novel solutions. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 371(1690) (2016).
  22. 22.
    Rosen, S.: The economics of superstars. Am. Econ. Rev. 71(5), 845–858 (1981)Google Scholar
  23. 23.
    Salganik, M.J., Dodds, P.S., Watts, D.: Experimental study of inequality and unpredictability in an artificial cultural market. Science 311(10), 854–856 (2006)CrossRefGoogle Scholar
  24. 24.
    Schilling, M.A.: A “small-world” network model of cognitive insight. Creativity Res. J. 17(2–3), 131–154 (2005).
  25. 25.
    Spitz, A., Horvát, E.Á.: Measuring long-term impact based on network centrality: unraveling cinematic citations. PloS one 9(10), e108857 (2014)Google Scholar
  26. 26.
    Thelwall, M., Maflahi, N.: When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals. J. Assoc. Inf. Sci. Technol. (2014)Google Scholar
  27. 27.
    Uzzi, B., Mukherjee, S., Stringer, M., Jones, B.: Atypical combinations and scientific impact. Science 342(6157), 468–472 (2013). CrossRefGoogle Scholar
  28. 28.
    Wang, D., Song, C., Barabási, A.L.: Quantifying long-term scientific impact. Science 342(6154), 127–132 (2013). CrossRefGoogle Scholar
  29. 29.
    Wasserman, M., et al.: Correlations between user voting data, budget, and box office for films in the internet movie database. J. Assoc. Inf. Sci. Technol. 66(4), 858–868 (2015). CrossRefGoogle Scholar
  30. 30.
    Wasserman, M., Zeng, X.H.T., Amaral, L.A.N.: Cross-evaluation of metrics to estimate the significance of creative works. Proc. Natl. Acad. Sci. 112(5), 1281–1286 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Watts, D.: Everything is Obvious: Once You Know the Answer. Crown Business (2011)Google Scholar
  32. 32.
    Weitzman, M.: Recombinant growth. Q. J. Econ. 113(2) (1998)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Northwestern UniversityEvanstonUSA
  2. 2.NRU Higher School of EconomicsMoscowRussia

Personalised recommendations