This paper discusses and copes with the difficulties that arise when trying to reproduce the results of the Shanghai academic ranking of world universities. In spite of the ambiguity of the methodology of the ranking with regard to the computation of the scores on its six indicators, the paper presents a set of straightforward procedures to estimate raw results and final relative scores. Discrepancies between estimated scores and the results of the ranking are mostly associated with the difficulties encountered in the identification of institutional affiliations, and are not significant. We can safely state that the results of the Shanghai academic ranking of world universities are in fact reproducible.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Australian Government (2011) Australian Government, Department of Education, Employment and Workplace Relations: Staff 2010 Full Time Equivalence. Web address: http://www.deewr.gov.au/HigherEducation/Publications/HEStatistics/Publications/Pages/Staff.aspx.
Billaut, J. C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shangai ranking: an MCDM view. Scientometrics, 84(1), 237–263.
Castells, M. (2005). The Network Society: From Knowledge to Policy, chap The Network Society: From Knowledge to Policy (pp. 3–22). Washington: Johns Hopkins Center for Transatlantic Relations.
Dehon, C., McCathie, A., & Verardi, V. (2010). Uncovering excellence in academic rankings: a closer look at the Shanghai ranking. Scientometrics, 83(2), 515–524.
Docampo, D. (2008). International rankings and quality of the university systems. Revista de Educación, Special Issue, 149–176.
Docampo, D. (2011). On using the Shanghai ranking to assess the research performance of university systems. Scientometrics, 86(1), 77–92.
Docampo, D. (2012). Universities from Spain in the 2011 edition of the Shanghai ranking. Submitted to Revista de Educación. Working paper available at http://www.gts.tsc.uvigo.es/∼ddocampo/ONLINE/docamporevistaeducacion.pdf.
Florian, R. V. (2007). Irreproducibility of the results of the Shangai academic ranking of world universities. Scientometrics, 72(1), 25–32.
Hazeltorn, E. (2008). Learning to live with league tables and ranking: the experience of institutional leaders. Higher Education Policy, 21, 193–215.
Himanen, P. (2005). The Network Society: From Knowledge to Policy, chap Challenges of the Global Information Society (pp. 337–372). Washington, DC: Johns Hopkins Center for Transatlantic Relations.
Ioannidis, J. P. A., Patsopoulos, N. A., Kavvoura, F. K., Tatsioni, A., Evangelou, E., Kouri, I., Contapoulos Ioannidis, D. G., & Liberopoulos, G. (2007). International ranking systems for universities and institutions: a critical appraisal. BMC Medicine, 5:30.
Kivinen, O., & Hedman, J. (2008). World-wide university rankings: a Scandinavian approach. Scientometrics, 74(3), 391–408.
Leyersdoff, L. (2012). World shares of publications of the USA EU-27 and China compared and predicted using the new Web of Science interface versus Scopus. El profesional de la información, 21(1), 43–49.
Liu, N. C., & Cheng, Y. (2005). Academic ranking of world universities: Methodologies and problems. Higher Education in Europe, 30(2), 127–136.
Liu, N. C., Cheng, Y., & Liu, L. (2005). Academic ranking of world universities using scientometrics: A comment to the “fatal attraction”. Scientometrics, 64(1), 101–109.
Marginson, S. (2005) There must be some way out of here. Tertiary Educ. Management Conference. Perth: Keynote address.
Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate Statistics, 5th edn). Boston: Pearson Education, Inc., Allyn and Bacon.
Van Parijs, P. (2009). European higher education under the spell of rankings. Ethical Perspectives, 16(2), 189–206.
Van Raan, A. F. J. (2005a). Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.
Van Raan, A. F. J. (2005b). Reply to the comments of Liu et al. Scientometrics, 64(1), 111–112.
Zitt, M., & Filliatreau, G. (2007). The World Class Universities and Ranking: Siming Beyond Status, chap Big is (made) beautiful: Some comments about the Shangai ranking of world-class universities, Part Two, 141–160. Romania: UNESCO-CEPES, Cluj University Press.
Additional Data sources
About this article
Cite this article
Docampo, D. Reproducibility of the Shanghai academic ranking of world universities results. Scientometrics 94, 567–587 (2013). https://doi.org/10.1007/s11192-012-0801-y