Higher Education

, Volume 60, Issue 5, pp 507–528 | Cite as

Ranking of rankings: benchmarking twenty-five higher education ranking systems in Europe

  • Ingo Stolz
  • Darwin D. HendelEmail author
  • Aaron S. Horn


The purpose of this study is to evaluate the ranking practices of 25 European higher education ranking systems (HERSs). Ranking practices were assessed with 14 quantitative measures derived from the Berlin Principles on Ranking of Higher Education Institutions (BPs). HERSs were then ranked according to their degree of congruence with the BPs. Additionally, the three domains of methodology, transparency, and consumer-friendliness were proposed to underlie the BPs, and the measures were also applied for assessing HERSs regarding these domains. Results indicate that the cooperating HERSs by CHE (Germany), AQA (Austria) and swissUp (Switzerland) exhibit the highest congruence with the BPs. However, no European HERS demonstrates good overall congruence with the BPs, mainly due to methodological shortcomings. Results further show that HERSs compiled and published by non-profit research entities seem to perform better than the majority of HERSs published by for-profit news outlets. International comparisons indicate that HERSs published in German-speaking countries and the Netherlands tend to exhibit a higher congruence with the BPs. Overall, this study hopes to stimulate the improvement of ranking practices through benchmarking with existing exemplary models. The quantitative assessment tool further promises to be useful in explaining relative stability or change of higher education institutions in HERSs, as well as in helping to explain resource allocation pressures within higher education institutions.


University ranking Higher education Berlin Principles Europe Comparative Benchmarking 


  1. Alexander von Humboldt-Stiftung. (2005). Das Humboldt-Ranking. Accessed December 29 2006.
  2. Altbach, P. G. (2003). The costs and benefits of world class universities. International Higher Education, 33. Accessed September 27 2008.
  3. Altbach, P. G. (2006). The dilemmas of ranking. International Higher Education, 42, 2–4. Accessed September 27 2008.
  4. ARRA. (2005). Assessment of public universities and their faculties. Accessed December 13 2006.
  5. Bellon, B. (2007). Evaluation practices and methodologies: Lessons for university ranking. In J. Sadlak & N. C. Liu (Eds.), The world-class university and ranking: Aiming beyond status (pp. 123–145). Bucharest, Rumania: UNESCO-CEPES.Google Scholar
  6. Berghoff, S., Federkeil, G., Giebisch, P., Hachmeister, C.-D., Hennings, M., & Müller-Böling, D. (2007). CHE—HochschulRanking: Vorgehensweise und Indikatoren—2007. Guetersloh, Germany: Centrum für Hochschulentwicklung.Google Scholar
  7. Birnbaum, R. (2006). No world-class university left behind. International Higher Education, 47. Accessed September 27 2008.
  8. Bowden, R. (2000). Fantasy higher education: University and college league tables. Quality in Higher Education, 6(1), 47–60.CrossRefGoogle Scholar
  9. Breimer, D. (2007). Rankings: How should universities respond? Paper presented at the 2nd Leiden University International Symposium on Ranking, Leiden, the Netherlands.Google Scholar
  10. Brooks, R. L. (2005). Measuring university quality. The Review of Higher Education, 29(1), 1–21.CrossRefGoogle Scholar
  11. CEST. (2004). CEST scientometrics research portfolios: Universities and colleges participating in the champions league—Diagrams and profiles, 1998–2002. Bern, Switzerland: Center for Science and Technology Studies of the Swiss Federal Government.Google Scholar
  12. Cheng, Y., & Liu, N. C. (2008). Examining major rankings according to the Berlin Principles. Higher Education in Europe, 33(2/3), 201–208.CrossRefGoogle Scholar
  13. Clarke, M. (2002). Some guidelines for academic quality rankings. Higher Education in Europe, 27(4), 443–459.CrossRefGoogle Scholar
  14. de Miguel, J., Cais, J., & Vaquera, E. (2001). Excelencia: Calidad de las universidades espanolas. Madrid, Spain: Centro de Investigaciones Sociologicas.Google Scholar
  15. de Miguel, J., Vaquera, E., & Sanchez, J. (2005). Spanish universities and the ranking 2005 initiative. Higher Education in Europe, 30(2), 199–215.CrossRefGoogle Scholar
  16. de Wit, K. (2003). The consequences of European integration for higher education. Higher Education Policy, 16(2), 161–178.CrossRefGoogle Scholar
  17. Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems. Higher Education, 49(4), 495–533.CrossRefGoogle Scholar
  18. Elsevier. (2006, October). Studeren. Elsevier Thema.Google Scholar
  19. Enserink, M. (2007). Who ranks the university rankers? Science, 317(5841), 1026–1028.CrossRefGoogle Scholar
  20. Federkeil, G. (2008). Rankings and quality assurance in higher education. Higher Education in Europe, 33(2/3), 219–231.CrossRefGoogle Scholar
  21. Gaertner, R., Birke, B., & Kohler, A. (2007). Hochschulranking 2007—Ingenieurwissenschaften, Geisteswissenschaften, Erziehungswissenschaften und Psychologie: Bericht zu Ergebnissen der österreichischen Universitäten, Verfahren und Methodik. Vienna, Austria: Austrian Agency for Quality Assurance.Google Scholar
  22. Hazelkorn, E. (2007). The impact of league tables and ranking systems on higher education decision making. Higher Education Management and Policy, 29(2), 87–110.Google Scholar
  23. Hendel, D. D., & Stolz, I. (2008). A comparative analysis of higher education ranking systems in Europe. Tertiary Education and Management, 14(3), 173–189.CrossRefGoogle Scholar
  24. IHEP. (2007). College and university ranking systems: Global perspectives and American challenges. Washington, DC: Institute for Higher Education Policy.Google Scholar
  25. IREG. (2006). Berlin Principles on Ranking of Higher Education Institutions. Berlin, Germany: International Ranking Expert Group.Google Scholar
  26. Karlsson, S., & Wadskog, D. (2006). Hur mycket citeras Svenska publikationer? Bibliometrisk översikt över Sveriges vetenskapliga publicering mellan 1982 och 2004. Stockholm, Sweden: Vetenskapsrådet.Google Scholar
  27. Lebherz, C., Mohr, C., Henning, M., & Sedlmeier, P. (2005). Wie brauchbar sind Hochschul-Rankings? Eine empirische Analyse. In U. Teichler & R. Tippelt (Eds.), Hochschullandschaft im Wandel (pp. 188–208). Germany; Switzerland, Beltz: Weinheim; Basel.Google Scholar
  28. Longden, B., & Yorke, M. (2006). Institutional rankings, marketing, and the needs of intending students. Paper presented at the Annual EAIR Forum, Rome, August 30 to September 1.Google Scholar
  29. Macleod, D. (2007). The guardian university guide 2008. London, UK: Guardian Newspapers.Google Scholar
  30. Marginson, S. (2007). Global university comparison: the second stage. Griffith University/IRU symposium on international trends in university rankings and classifications. Accessed May 15 2008.
  31. Marginson, S., & van der Wende, M. (2007). To rank or to be ranked: The impact of global rankings in higher education. Journal of Studies in International Education, 11(3/4), 306–329.CrossRefGoogle Scholar
  32. McCormick, A. C. (2008). The complex interplay between classification and ranking of colleges and universities: Should the Berlin Principles apply equally to classification? Higher Education in Europe, 33(2/3), 209–218.CrossRefGoogle Scholar
  33. McDonough, P. M., Lising, A., Walpole, A. M., & Perez, L. X. (1998). College rankings: Democratized college knowledge for whom? Research in Higher Education, 39(5), 513–537.CrossRefGoogle Scholar
  34. Merisotis, J. P. (2002). Summary report of the invitational roundtable on statistical indicators for the quality assessment of higher/tertiary education institutions: Ranking and league table methodologies. Higher Education in Europe, 27(4), 476–480.Google Scholar
  35. Merisotis, J., & Sadlak, J. (2005). Higher education rankings: Evolution, acceptance, and dialogue. Higher Education in Europe, 30(2), 97–101.CrossRefGoogle Scholar
  36. Müller-Böling, D. (2001a). Hochschulranking—Aussagefähigkeit, Methoden, Probleme: Eine Einführung. In D. Müller-Böling, S. Hornbostel, & S. Berghoff (Eds.), Hochschulranking: Aussagefähigkeit, Methoden, Probleme; wissenschaftliche Tagung Berlin 25/26. September 2000 (pp. 5–6). Guetersloh, Germany: Bertelsmann-Stiftung.Google Scholar
  37. Müller-Böling, D. (2001b). Qualität an deutschen Hochschulen: Der Beitrag des Hochschulrankings. In Schweizerische Akademie der Geistes- und Sozialwissenschaften (Ed.), Welche Qualität in den Sozialwissenschaften? Ansätze, Erfahrungen, Perspektiven: Eine Tagung des Wissenschaftspolitischen Rates für Sozialwissenschaften (CPS/WRS), des Schwerpunktprogramms ‘Zukunft Schweiz’, der Schweizerischen Evaluationsgesellschaft SEVAL und der Schweizerischen Akademie der Geistes- und Sozialwissenschaften, Gurten/Bern, 16./17. Mai 2001 (pp. 125–142). Bern, Switzerland: Schweizerische Akademie der Geistes- und Sozialwissenschaften.Google Scholar
  38. O’Leary, J., Hindmarsh, A., & Kingston, B. (2006a). The Times Good University Guide 2007. London, UK: Times Books.Google Scholar
  39. O’Leary, J., Quacquarelli, N., & Ince, M. (2006b). Guide to world’s top universities: Exclusively featuring the complete THES/QS world university rankings. London, UK: QS Quacquarelli Symonds, Blackwell Publishing.Google Scholar
  40. PRIME. (2006). Observatory of the European university: Methodological guide. Lugano, Switzerland: PRIME Network of Excellence.Google Scholar
  41. Proulx, R. (2006). World university rankings: The need for a new paradigm. Paper presented at the 28th annual EAIR Forum, Rome, August 30–September 1.Google Scholar
  42. Sadlak, J., Merisotis, J., & Liu, N. C. (2008). University rankings: Seeking prestige, raising visibility and embedding quality—The editor’s view. Higher Education in Europe, 33(2/3), 195–199.CrossRefGoogle Scholar
  43. Salmi, J., & Saroyan, A. (2007). League tables as policy instruments: Uses and misuses. Higher Education Management & Policy, 19(2), 31–68.Google Scholar
  44. Steenkamp, F., de Looper, H., & Bliekendaal, M. (2006). Keuzegids Hoger Onderwijs 2006–2007: Verzamelgids. Leiden, The Netherlands: Keuzegids Hoger Onderwijs.Google Scholar
  45. Studentenspiegel. (2006). Methodik des Studentenspiegels. Accessed February 14 2007.
  46. swissUp. (2006a). Definitionen der gerankten Indikatoren: Ranking 2005 + 2006. Accessed January 29 2007.
  47. swissUp. (2006b). swissUp Ranking 2006, Bemerkungen zur Methodologie. Accessed January 29 2007.
  48. Sydsvenska Industri-och Handelskammaren. (2007). Högskolerankingen. Accessed May 14 2008.
  49. Taylor, P., & Braddock, R. (2007). International university ranking systems and the idea of university excellence. Journal of Higher Education Policy and Management, 29(3), 245–260.CrossRefGoogle Scholar
  50. Terenzini, P. T., & Pascarella, B. T. (1994). Living with myths: Undergraduate education and what we can do about it. Change, 26(1), 28–32.Google Scholar
  51. UNESCO-CEPES. (2006). Second meeting of the International Rankings Expert Group: List of participants. Accessed October 3 2008.
  52. Usher, A., & Savino, M. (2006). A world of difference: A global survey of university league tables. Toronto, Canada: Educational Policy Institute.Google Scholar
  53. Usher, A., & Savino, M. (2007). A global survey of university ranking and league tables. Higher Education in Europe, 32(1), 5–15.CrossRefGoogle Scholar
  54. van Dyke, N. (2005). Twenty years of university reports cards. Higher Education in Europe, 30(2), 103–124.CrossRefGoogle Scholar
  55. Yorke, M. (1997). A good league table guide? Quality Assurance in Education, 5(2), 61–72.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  1. 1.Department of Organizational Leadership, Policy, and DevelopmentUniversity of MinnesotaMinneapolisUSA

Personalised recommendations