, Volume 94, Issue 2, pp 567–587 | Cite as

Reproducibility of the Shanghai academic ranking of world universities results

  • Domingo DocampoEmail author


This paper discusses and copes with the difficulties that arise when trying to reproduce the results of the Shanghai academic ranking of world universities. In spite of the ambiguity of the methodology of the ranking with regard to the computation of the scores on its six indicators, the paper presents a set of straightforward procedures to estimate raw results and final relative scores. Discrepancies between estimated scores and the results of the ranking are mostly associated with the difficulties encountered in the identification of institutional affiliations, and are not significant. We can safely state that the results of the Shanghai academic ranking of world universities are in fact reproducible.


Shanghai Ranking Reproducibility Results ARWU 


  1. Australian Government (2011) Australian Government, Department of Education, Employment and Workplace Relations: Staff 2010 Full Time Equivalence. Web address:
  2. Billaut, J. C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shangai ranking: an MCDM view. Scientometrics, 84(1), 237–263.CrossRefGoogle Scholar
  3. Castells, M. (2005). The Network Society: From Knowledge to Policy, chap The Network Society: From Knowledge to Policy (pp. 3–22). Washington: Johns Hopkins Center for Transatlantic Relations.Google Scholar
  4. Dehon, C., McCathie, A., & Verardi, V. (2010). Uncovering excellence in academic rankings: a closer look at the Shanghai ranking. Scientometrics, 83(2), 515–524.CrossRefGoogle Scholar
  5. Docampo, D. (2008). International rankings and quality of the university systems. Revista de Educación, Special Issue, 149–176.Google Scholar
  6. Docampo, D. (2011). On using the Shanghai ranking to assess the research performance of university systems. Scientometrics, 86(1), 77–92.CrossRefGoogle Scholar
  7. Docampo, D. (2012). Universities from Spain in the 2011 edition of the Shanghai ranking. Submitted to Revista de Educación. Working paper available at∼ddocampo/ONLINE/docamporevistaeducacion.pdf.
  8. Florian, R. V. (2007). Irreproducibility of the results of the Shangai academic ranking of world universities. Scientometrics, 72(1), 25–32.MathSciNetCrossRefGoogle Scholar
  9. Hazeltorn, E. (2008). Learning to live with league tables and ranking: the experience of institutional leaders. Higher Education Policy, 21, 193–215.CrossRefGoogle Scholar
  10. Himanen, P. (2005). The Network Society: From Knowledge to Policy, chap Challenges of the Global Information Society (pp. 337–372). Washington, DC: Johns Hopkins Center for Transatlantic Relations.Google Scholar
  11. Ioannidis, J. P. A., Patsopoulos, N. A., Kavvoura, F. K., Tatsioni, A., Evangelou, E., Kouri, I., Contapoulos Ioannidis, D. G., & Liberopoulos, G. (2007). International ranking systems for universities and institutions: a critical appraisal. BMC Medicine, 5:30.CrossRefGoogle Scholar
  12. Kivinen, O., & Hedman, J. (2008). World-wide university rankings: a Scandinavian approach. Scientometrics, 74(3), 391–408.CrossRefGoogle Scholar
  13. Leyersdoff, L. (2012). World shares of publications of the USA EU-27 and China compared and predicted using the new Web of Science interface versus Scopus. El profesional de la información, 21(1), 43–49.CrossRefGoogle Scholar
  14. Liu, N. C., & Cheng, Y. (2005). Academic ranking of world universities: Methodologies and problems. Higher Education in Europe, 30(2), 127–136.CrossRefGoogle Scholar
  15. Liu, N. C., Cheng, Y., & Liu, L. (2005). Academic ranking of world universities using scientometrics: A comment to the “fatal attraction”. Scientometrics, 64(1), 101–109.CrossRefGoogle Scholar
  16. Marginson, S. (2005) There must be some way out of here. Tertiary Educ. Management Conference. Perth: Keynote address.Google Scholar
  17. Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate Statistics, 5th edn). Boston: Pearson Education, Inc., Allyn and Bacon.Google Scholar
  18. Van Parijs, P. (2009). European higher education under the spell of rankings. Ethical Perspectives, 16(2), 189–206.CrossRefGoogle Scholar
  19. Van Raan, A. F. J. (2005a). Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.CrossRefGoogle Scholar
  20. Van Raan, A. F. J. (2005b). Reply to the comments of Liu et al. Scientometrics, 64(1), 111–112.CrossRefGoogle Scholar
  21. Zitt, M., & Filliatreau, G. (2007). The World Class Universities and Ranking: Siming Beyond Status, chap Big is (made) beautiful: Some comments about the Shangai ranking of world-class universities, Part Two, 141–160. Romania: UNESCO-CEPES, Cluj University Press.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2012

Authors and Affiliations

  1. 1.Universidad de Vigo, Atlantic Research Center for Information and Communication TechnologiesVigoSpain

Personalised recommendations