Scientometrics

, Volume 85, Issue 1, pp 243–256 | Cite as

Comparing university rankings

  • Isidro F. Aguillo
  • Judit Bar-Ilan
  • Mark Levene
  • José Luis Ortega
Article

Abstract

Recently there is increasing interest in university rankings. Annual rankings of world universities are published by QS for the Times Higher Education Supplement, the Shanghai Jiao Tong University, the Higher Education and Accreditation Council of Taiwan and rankings based on Web visibility by the Cybermetrics Lab at CSIC. In this paper we compare the rankings using a set of similarity measures. For the rankings that are being published for a number of years we also examine longitudinal patterns. The rankings limited to European universities are compared to the ranking of the Centre for Science and Technology Studies at Leiden University. The findings show that there are reasonable similarities between the rankings, even though each applies a different methodology. The biggest differences are between the rankings provided by the QS-Times Higher Education Supplement and the Ranking Web of the CSIC Cybermetrics Lab. The highest similarities were observed between the Taiwanese and the Leiden rankings from European universities. Overall the similarities are increased when the comparison is limited to the European universities.

Keywords

Ranking Universities Shanghai ranking Times ranking Taiwan ranking Leiden ranking Webometrics ranking Comparative analysis 

References

  1. Aguillo, I. F., Granadino, B., Ortega, J. L., & Prieto, J. A. (2006). Scientific research activity and communication measured with cybermetric indicators. Journal of the American Society of Information Science and Technology, 57(10), 1296–1302.CrossRefGoogle Scholar
  2. Aguillo, I. F., Ortega, J. L., & Fernández, M. (2008). Webometric ranking of world universities: Introduction, methodology, and future developments. Higher Education in Europe, 33(2/3), 234–244.Google Scholar
  3. Bar-Ilan, J., Levene, M., & Lin, A. (2007). Some measures for comparing citation databases. Journal of Informetrics, 1, 26–34.CrossRefGoogle Scholar
  4. Billaut, J., Bouyssou, D., & Vinke, P. (2010). Should we believe the Shanghai ranking? An MCDM view. Scientometrics. doi:10.1007/s11192-009-0115-x.
  5. Black, T. R. (1994). Evaluating social science research: An introduction. London: SAGE Publications.Google Scholar
  6. CEPES. (2006). The Berlin principles on ranking of higher education institutions. Retreived January 11, 2009 from http://www.cepes.ro/hed/meetings/berlin06/Berlin%20Principles.pdf.
  7. Diaconis, P., & Graham, R. L. (1977). Spearman’s footrule as a measure of disarray. Journal of the Royal Statistical Society. Series B (Methodological), 39, 262–268.MATHMathSciNetGoogle Scholar
  8. Dill, D. D., & Soo, M. (2005). Academic quality, league tables and public policy: A cross national analysis of university ranking systems. Higher Education, 49, 499–533.CrossRefGoogle Scholar
  9. Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001). Rank aggregation methods for the Web. In Proceedings of the 10th World Wide Web Conference, May 2001, Hong-Kong, (pp. 613–622).Google Scholar
  10. Eccles, C. (2002). The use of university rankings in the United Kingdom. Higher Education in Europe, 27(4), 423–432.CrossRefGoogle Scholar
  11. Fagin, R., Kumar, R., & Sivakumar, D. (2003). Comparing top k lists. SIAM Journal on Discrete Mathematics, 17(1), 134–160.MATHCrossRefMathSciNetGoogle Scholar
  12. Liu, N. C., & Cheng, Y. (2005). The academic ranking of world universities—methodologies and problems. Higher Education in Europe, 30(2), 127–136.CrossRefGoogle Scholar
  13. Liu, N. C., & Cheng, Y. (2008). Examining major rankings according to the Berlin principles. Higher Education in Europe, 33(2/3), 201–208.Google Scholar
  14. Marginson, S., & van der Wende, M. (2007). To rank or to be ranked: The impact of global rankings in higher education. Journal of Studies in International Education, 11(3/4), 306–329.CrossRefGoogle Scholar
  15. van Raan, A. F. J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36, 397–420.CrossRefGoogle Scholar
  16. van Raan, A. F. J. (2005). Fatal attraction—conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.CrossRefGoogle Scholar
  17. Webster, T. J. (2001). A principal component analysis of the US News & World Report tier rankings of colleges and universities. Economics of Education Review, 20(3), 235–244.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2010

Authors and Affiliations

  • Isidro F. Aguillo
    • 1
  • Judit Bar-Ilan
    • 2
  • Mark Levene
    • 3
  • José Luis Ortega
    • 4
  1. 1.Cybermetrics LabCCHS – CSICMadridSpain
  2. 2.Department of Information ScienceBar-Ilan UniversityRamat GanIsrael
  3. 3.School of Computer Science and Information SystemsBirkbeck University of LondonLondonUK
  4. 4.Scientific Programming DivisionVICYT – CSICMadridSpain

Personalised recommendations