Skip to main content
SpringerLink
Log in

The Informative Value of International University Rankings: Some Methodological Remarks

  • Chapter
  • First Online:
Incentives and Performance

Abstract

Global university rankings are critically discussed concerning their outcome, e.g., pointing to contradictory results. In contrast, this paper examines three issues with the example of three well known rankings (Shanghai, Times and Taiwan). First, on which methods, in particular on which indicators are they based? Second, are there fundamental deficits of concept and systematic biases? Third, can one predict good or bad ranking scores, based on the respective concept? In addition with the example of German universities there are two factors on performance measurement discussed, disciplinary cultures and size of the university. In sum, the usefulness of global rankings is considered to be highly questionable. Measures in science policy should not be legitimated with such rankings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    320 citations in Google Scholar on 16 January 2014.

  2. 2.

    http://www.timeshighereducation.co.uk/world-university-rankings/2013-14/subject-ranking/subject/arts-and-humanities/methodology

  3. 3.

    Defined by the location of the headquarter of the publisher.

  4. 4.

    Examples of a normalization are discussed in Sect. 3.1.

  5. 5.

    In its latest version.

  6. 6.

    CWTS (Centrum voor Wetenshap- en Techologie Studies, engl. Center for Science and Technology Studies), Leiden, Netherlands is a leading research center in bibliometrics.

  7. 7.

    For a more detailed explanation of the SR index cf. Michels and Schmoch (2014) and Schmoch et al. (2012).

  8. 8.

    Download from http://datahub.io/de/dataset/eumida

References

  • Aguillo IF, Bar-Ilan J, Levene M et al (2010) Comparing university rankings. Scientometrics 85(1):243–256. doi:10.1007/s11192-010-0190-z

    Article  Google Scholar 

  • Bowman NA, Bastedo MN (2011) Anchoring effects in world university rankings: exploring biases in reputation scores. Higher Educ 61(4):431–444

    Article  Google Scholar 

  • Dill DD, So M (2005) Academic quality, league tables, and public policy: a cross-national analysis of university ranking systems. Higher Educ 49(4):495–533

    Article  Google Scholar 

  • Florian RV (2007) Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics 72(1):25–32

    Article  Google Scholar 

  • Grupp H, Mogee ME (2004) Indicators for national science and technology policy: how robust are composite indicators? Res Policy 33(9):1373–1384

    Article  Google Scholar 

  • Grupp H, Schmoch U, Hinze S (2001) International alignment and scientific regard as macro-indicators for international comparisons of publications. Scientometrics 51(2):359–380

    Article  Google Scholar 

  • Hazelkorn E (2011) How rankings are reshaping higher education. In: Climent V, Michavila F, Ripolles M (eds) Los rankings univeritarios: Mitos y Realidades, Ed. Tecnos

    Google Scholar 

  • Hicks D (2004) The four literatures of social science. In: Moed H, Glaenzel W, Schmoch U (eds) Handbook of quantitative science and technology research. Kluwer, Dordrecht, pp 473–496

    Google Scholar 

  • Ioannidis JPA, Nikolaos A, Patsopoulos NA, Fotini K et al (2007) International ranking systems for universities and institutions: a critical appraisal. BMC Med 5:30. doi:10.1186/1741-7015-5-30

    Article  Google Scholar 

  • Ishikawa M (2009) University rankings, global models, and emerging hegemony. Critical analysis from Japan. J Stud Int Educ 13(2):159–173. doi:10.1177/1028315308330853

    Article  Google Scholar 

  • Kehm BM (2013) Der Kampf ums Treppchen. Wie Hochschulen bei der Lieferung von Daten für internationale Rankings schummeln. Paper presented at the Fraunhofer ISI workshop “Perspektiven das deutsche Hochschulsystem”, Berlin, 21–22 Nov 2013. Acessible through http://www.isi.fraunhofer.de/isi-de/p/veranstaltungen/ws_ho-forschung_11-2013.php

  • Kehm BM (2014) Global university rankings—impacts and unintended side effects. Eur J Educ 49(1):102–112

    Article  Google Scholar 

  • Liu NC, Cheng Y (2005) The academic ranking of world universities. Higher Educ Eur 30(2):127–136. doi:10.1080/03797720500260116Scopus76

  • Marginson S (2007) Global university rankings: implications in general and for Australia. J Higher Educ Policy Manage 29(2):131–142

    Article  Google Scholar 

  • Marginson S, van der Wende M (2007) To rank or to be ranked: the impact of global rankings in higher education. J Stud Int Educ 11(3–4):306–329. doi:10.1177/1028315307303544

    Article  Google Scholar 

  • Michels C, Schmoch U (2014) Impact of bibliometric studies on the publication behaviour of authors. Scientometrics 98(1):369–385

    Article  Google Scholar 

  • Mueller PS, Murali NS, Cha SS, Erwin PJ, Ghosh AK (2006) The association between impact factors and language of general internal medicine journals. Swiss Med Wkly 136:441–443

    Google Scholar 

  • Mu-Hsuan H (2011) A comparison of three major academic rankings for world universities: from a research evaluation perspective. J Libr Inf Stud 9(1):1–25

    Google Scholar 

  • Rauhvargers A (2011) Global university rankings and their impact. Leadership for WorldClass Universities. Challenges for Developing Countries, No. European University Association, Brussels

    Google Scholar 

  • Schmoch U, Mallig N, Michels C, Neuhäusler P, Schulze N (2011) Performance and structures in the German science system in an international comparison 2010. Studien zum deutschen Innovationssystem 8-2011. EFI, Berlin

    Google Scholar 

  • Schmoch U, Michels C, Schulze N, Neuhäusler P (2012) Performance and structures of the German science system 2011. Germany in an international comparison, China’s profile, behaviour of German authors, comparison of Web of Science and Scopus. Studien zum deutschen Innovationssystem Nr. 9–2012. EFI, Berlin

    Google Scholar 

  • Schubert T, Michels C (2013) Placing articles in the large publisher nations: is there a “free lunch” in terms of higher impact? J Am Soc Inf Sci Technol 64(3):596–611

    Article  Google Scholar 

  • Teichler U (2011) Social contexts and systemic consequences of university rankings: a meta-analysis of the ranking literature. In: Shin JC, Toutkoushian RK, Teichler U (eds) University rankings. Theoretical basis, methodology and impact on global higher education. Springer, Dordrecht, pp 55–69

    Google Scholar 

  • van Raan AFJ (2005) Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics 62(1):133–143. doi:10.1007/s11192-005-0008-6

    Article  Google Scholar 

  • van Raan AFJ, van Leeuwen TN, Martijn S, Visser MS (2011) Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings. Scientometrics 88(2):495–498

    Article  Google Scholar 

  • van Vught FA, Ziegele F (eds) (2012) Multidimensional ranking. The design and development of U-multirank. Springer, Dordrecht

    Google Scholar 

  • Ziegele F, van Vught F (2013) “U-Multirank” und “U-Map” als Ansätze zur Schaffung von Transparenz im europäischen und globalen Hochschulsystem – Konzepte und Erfahrungen. Beiträge zur Hochschulforschung 35(2):50–74

    Google Scholar 

Download references

Acknowledgement

I thank Nicole Schulze for the careful conduct of the bibliometric analysis and for various helpful comments.

Certain data included herein is derived from the Science Citation Index, Social Science Citation Index and Arts & Humanities Citation Index databases prepared by Thomson Reuters (Scientific) Inc. (TR®), Philadelphia, PA, USA: ©Copyright Thomson Reuters (Scientific) 2010. All rights reserved.

Author information

Authors and Affiliations

  1. University of Administrative Sciences, Speyer, Germany

    Ulrich Schmoch

  2. Fraunhofer Institute for Systems and Innovation Research, Karlsruhe, Germany

    Ulrich Schmoch

Authors
  1. Ulrich Schmoch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ulrich Schmoch .

Editor information

Editors and Affiliations

  1. TUM School of Management, Technische Universität München, Munich, Germany

    Isabell M. Welpe

  2. TUM School of Management, Technische Universität München, Munich, Germany

    Jutta Wollersheim

  3. TUM School of Management, Technische Universität München, Munich, Germany

    Stefanie Ringelhan

  4. Zeppelin University, Friedrichshafen, Germany, and University of Zurich, Zurich, Switzerland

    Margit Osterloh

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Schmoch, U. (2015). The Informative Value of International University Rankings: Some Methodological Remarks. In: Welpe, I., Wollersheim, J., Ringelhan, S., Osterloh, M. (eds) Incentives and Performance. Springer, Cham. https://doi.org/10.1007/978-3-319-09785-5_9

Download citation

Publish with us

Policies and ethics

Search

Navigation