Advertisement

An Evaluation and Critique of Current Rankings

  • Gero Federkeil
  • Frans A. van Vught
  • Don F. Westerheijden
Chapter
Part of the Higher Education Dynamics book series (HEDY, volume 37)

Abstract

This chapter raises the question of whether university league tables deliver relevant information to one of their key target groups – students. It examines the inherent biases and weaknesses in the methodologies of the major rankings and argues that the concentration on a single indicator of excellence (research) and single function of an institution ignores the diverse needs and motivations of prospective students in choosing a university. It also raises the issue of rankings that proclaim the ‘excellence’ of an entire institution, which may not be an accurate reflection of the performance of individual departments. The authors then present some principles and examples of good practice in ranking and discuss alternative classification systems.

Keywords

High Education Institution Composite Indicator League Table Global Ranking Integrate Postsecondary Education Data System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. AACSB. (2005). The business school rankings dilemma. A report from a task force of AACSB International’s Committee on Issues in Management Education. Tampa: AACSB.Google Scholar
  2. AUBR Expert Group. (2009). Assessing Europe’s university-based research – Draft. s. l. Brussels: European Commission – DG Research.Google Scholar
  3. Brennan, J., Goedegebuure, L. C. J., Shah, T., Westerheijden, D. F., & Weusthof, P. J. M. (1992). Towards a methodology for comparative quality assessment in European higher education: A pilot study on economics in Germany, the Netherlands and the United Kingdom. London/Enschede/Hannover: CNAA/CHEPS/HIS.Google Scholar
  4. Brown, R. (2006). League tables – Do we have to live with them? Perspectives, 10(2), 33–38.Google Scholar
  5. Cremonini, L., Westerheijden, D. F., & Enders, J. (2008). Disseminating the right information to the right audience: Cultural determinants in the use (and misuse) of rankings. Higher Education, 55, 373–385.CrossRefGoogle Scholar
  6. Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems. Higher Education, 49, 495–533.CrossRefGoogle Scholar
  7. Eccles, J. S., & Gootman, J. A. (2002). Community programs to promote youth development. Washington, DC: National Academy Press.Google Scholar
  8. Enserink, M. (2007). Who ranks the university rankers? Science, 317(5841), 1026–1028.CrossRefGoogle Scholar
  9. Filliatreau, G. & Zitt, M. (s. a.). Big is (made) Beautiful. Some comments about the Shanghai ranking of world-class universities. s. l. Paris: OST.Google Scholar
  10. Gottlieb, B. (2009). Cooking the School Books: How U.S. News cheats in picking its ‘best American colleges’. Slate, September 1, 2009.Google Scholar
  11. Hazelkorn, E. (2011). Rankings and the reshaping of higher education: The battle for world-class excellence. London: Palgrave Macmillan.CrossRefGoogle Scholar
  12. Högskolverket. (2009). Ranking of universities and higher education institutions for student information purposes? (No. 2009:27 R). Stockholm: Högskoleverket.Google Scholar
  13. International Ranking Expert Group. (2006). Berlin principles on ranking of higher education institutions. Retrieved June 24, 2006, from http://www.che.de/downloads/Berlin_Principles_IREG_534.pdf
  14. IREG Observatory (2011). IREG Ranking Audit Manual. IREG Observatory on Academic Rankings and Excellence, 2011. Available at: http://www.ireg-observatory.org/pdf/ranking_audith_audit.pdf
  15. King, R., Locke, W., Puncher, M., Richardson, J., & Verbik, L. (2008). Counting what is measured or measuring what counts? League tables and their impact on higher education institutions in England (No. April 2008/14). s. l. Higher Education Funding Council for England, Bristo.Google Scholar
  16. Klein, S. P., & Hamilton, L. (1998). The validity of the U. S. News & World Report ranking of ABA Law Schools. s. l. Association of American Law Schools.Google Scholar
  17. Leeuw, F. L. (2002). Evaluating and the ranking of higher education studies. In J. Bevers & M. Hulshof (Eds.), Willems & wetens, liber amicorum voor Jos Willems (pp. 141–154). Nijmegen: IOWO/KUN.Google Scholar
  18. Marginson, S. (2006, September 7–9). Global university rankings: private and public goods. Paper presented at the 19th Annual CHER conference, Kassel, Germany.Google Scholar
  19. Marginson, S. (2008). Global, multiple and engaged: Has the ‘Idea of a University’ changed in the era of the global knowledge economy? Paper presented at the fifth international workshop on Higher Education Reforms ‘The Internationalization of Higher Education and Higher Education Reforms’, Shanghai, China.Google Scholar
  20. Marginson, S. (2009). University rankings, government and social order: Managing the field of higher education according to the logic of the performative present-as-future. In M. Simons, M. Olssen, & M. Peters (Eds.), Re-reading education policies: Studying the policy agenda of the 21st century. Rotterdam: Sense Publishers.Google Scholar
  21. Merisotis, J. P. (2003). On the ranking of higher education institutions. Higher Education in Europe, 27(4), 361–364.CrossRefGoogle Scholar
  22. Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.Google Scholar
  23. Müller-Böling, D., & Federkeil, G. (2007). The CHE-Ranking of German, Swiss and Austrian Universities. In J. Sadlak & L. N. Cai (Eds.), The world-class university an ranking: Aiming beyond status (pp. 189–203). Bucharest: CEPES.Google Scholar
  24. National Opinion Research Center (1997). A review of the methodology for the U.S. news & world report’s rankings of undergraduate colleges and universities. The Washington Monthly Review Online. Retrieved January 23, 2012, from http://www.washingtonmonthly.com/features/2000/norc.html Google Scholar
  25. OECD (2008). Handbook on constructing composite indicators. Paris, OECD.Google Scholar
  26. Sadlak, J., & Liu, N. C. (2007). The world-class university and ranking: Aiming beyond status. Bucharest, Romania/ Shanghai, China/ Cluj-Napoca, Romania: UNESCO-CEPES.Google Scholar
  27. Saisana, M., & D’Hombres, B. (2008). Higher education rankings: Robustness issues and critical assessment. How much confidence can we have in higher education rankings? Luxembourg: Office for Official Publications of the European Communities.Google Scholar
  28. Salmi, J. (2009). The challenge of establishing world-class universities. Washington, D. C.: World Bank.CrossRefGoogle Scholar
  29. SCImago Research Group. (2009). SCImago Institutions Rankings (SIR): 2009 World Report. Retrieved November 11, 2009, from http://www.scimagoir.com/pdf/sir_2009_world_report.pdf
  30. Stigler, G. J. (1961). The economics of information. Journal of Political Economy, LXIX, 213–225.CrossRefGoogle Scholar
  31. Thibaud, A. (2009). Vers quel classement européen des universités? Etude comparative du classement de Shanghai et des autres classements internationaux (No. Note de Benchmarking 4). Brussels/Paris: Institut Thomas More.Google Scholar
  32. Usher, A., & Savino, M. (2006). A world of difference: A global survey of university league tables. Toronto: Educational Policy Institute.Google Scholar
  33. van der Wende, M. (2008). Rankings and classifications in higher education. A European perspective. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 23). Dordrecht: Springer.Google Scholar
  34. van der Wende, M., & Westerheijden, D. F. (2009). Rankings and classifications: The need for a multidimensional approach. In F. A. van Vught (Ed.), Mapping the higher education landscape: Towards a European classification of higher education (pp. 71–86). Dordrecht: Springer.Google Scholar
  35. Van Dyke, N. (2005). Twenty years of university report cards. Higher Education in Europe, 30(2), 103–125.CrossRefGoogle Scholar
  36. van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.CrossRefGoogle Scholar
  37. van Vught, F. A. (2008). Mission diversity and reputation in higher education. Higher Education Policy, 21(2), 151–174.CrossRefGoogle Scholar
  38. van Vught, F. A. (Ed.). (2009). Mapping the higher education landscape: Towards a European classification of higher education. Dordrecht: Springer.Google Scholar
  39. Vedder, R., & Ewalt, D. M. (2009). America’s best colleges. Forbes, August 5, 2009.Google Scholar
  40. Visser, M. S., & Moed, H. F. (2008). Comparing web of science and scopus on a paper-by-paper basis. Paper presented at the 10th international conference on Science & Technology Indicators, Vienna.Google Scholar
  41. Westerheijden, D. F. (2009). Information of quality and quality of information to match students and programme. In J. Newton & R. Brown (Eds.), The future of quality assurance. Amsterdam: EAIR.Google Scholar
  42. Westerheijden, D. F., Beerkens, E., Cremonini, L., Huisman, J., Kehm, B., Kovac, A., et al. (2010). The first decade of working on the European Higher Education Area: The Bologna Process Independent Assessment – Executive summary, overview and conclusions. s. l. Vienna: European Commission, Directorate-General for Education and Culture.Google Scholar
  43. Yorke, M. (1998). The Times’ league table of universities, 1997: A statistical appraisal. Quality Assurance in Education, 6(1), 58.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  • Gero Federkeil
    • 1
  • Frans A. van Vught
    • 2
  • Don F. Westerheijden
    • 2
  1. 1.Centre for Higher Education (CHE)GüterslohGermany
  2. 2.Center for Higher Education Policy StudiesUniversity of TwenteEnschedeThe Netherlands

Personalised recommendations