Comparing university rankings

Abstract

Recently there is increasing interest in university rankings. Annual rankings of world universities are published by QS for the Times Higher Education Supplement, the Shanghai Jiao Tong University, the Higher Education and Accreditation Council of Taiwan and rankings based on Web visibility by the Cybermetrics Lab at CSIC. In this paper we compare the rankings using a set of similarity measures. For the rankings that are being published for a number of years we also examine longitudinal patterns. The rankings limited to European universities are compared to the ranking of the Centre for Science and Technology Studies at Leiden University. The findings show that there are reasonable similarities between the rankings, even though each applies a different methodology. The biggest differences are between the rankings provided by the QS-Times Higher Education Supplement and the Ranking Web of the CSIC Cybermetrics Lab. The highest similarities were observed between the Taiwanese and the Leiden rankings from European universities. Overall the similarities are increased when the comparison is limited to the European universities.

This is a preview of subscription content, log in to check access.

References

  1. Aguillo, I. F., Granadino, B., Ortega, J. L., & Prieto, J. A. (2006). Scientific research activity and communication measured with cybermetric indicators. Journal of the American Society of Information Science and Technology, 57(10), 1296–1302.

    Article  Google Scholar 

  2. Aguillo, I. F., Ortega, J. L., & Fernández, M. (2008). Webometric ranking of world universities: Introduction, methodology, and future developments. Higher Education in Europe, 33(2/3), 234–244.

    Google Scholar 

  3. Bar-Ilan, J., Levene, M., & Lin, A. (2007). Some measures for comparing citation databases. Journal of Informetrics, 1, 26–34.

    Article  Google Scholar 

  4. Billaut, J., Bouyssou, D., & Vinke, P. (2010). Should we believe the Shanghai ranking? An MCDM view. Scientometrics. doi:10.1007/s11192-009-0115-x.

  5. Black, T. R. (1994). Evaluating social science research: An introduction. London: SAGE Publications.

    Google Scholar 

  6. CEPES. (2006). The Berlin principles on ranking of higher education institutions. Retreived January 11, 2009 from http://www.cepes.ro/hed/meetings/berlin06/Berlin%20Principles.pdf.

  7. Diaconis, P., & Graham, R. L. (1977). Spearman’s footrule as a measure of disarray. Journal of the Royal Statistical Society. Series B (Methodological), 39, 262–268.

    MATH  MathSciNet  Google Scholar 

  8. Dill, D. D., & Soo, M. (2005). Academic quality, league tables and public policy: A cross national analysis of university ranking systems. Higher Education, 49, 499–533.

    Article  Google Scholar 

  9. Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001). Rank aggregation methods for the Web. In Proceedings of the 10th World Wide Web Conference, May 2001, Hong-Kong, (pp. 613–622).

  10. Eccles, C. (2002). The use of university rankings in the United Kingdom. Higher Education in Europe, 27(4), 423–432.

    Article  Google Scholar 

  11. Fagin, R., Kumar, R., & Sivakumar, D. (2003). Comparing top k lists. SIAM Journal on Discrete Mathematics, 17(1), 134–160.

    MATH  Article  MathSciNet  Google Scholar 

  12. Liu, N. C., & Cheng, Y. (2005). The academic ranking of world universities—methodologies and problems. Higher Education in Europe, 30(2), 127–136.

    Article  Google Scholar 

  13. Liu, N. C., & Cheng, Y. (2008). Examining major rankings according to the Berlin principles. Higher Education in Europe, 33(2/3), 201–208.

    Google Scholar 

  14. Marginson, S., & van der Wende, M. (2007). To rank or to be ranked: The impact of global rankings in higher education. Journal of Studies in International Education, 11(3/4), 306–329.

    Article  Google Scholar 

  15. van Raan, A. F. J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36, 397–420.

    Article  Google Scholar 

  16. van Raan, A. F. J. (2005). Fatal attraction—conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.

    Article  Google Scholar 

  17. Webster, T. J. (2001). A principal component analysis of the US News & World Report tier rankings of colleges and universities. Economics of Education Review, 20(3), 235–244.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Isidro F. Aguillo.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Aguillo, I.F., Bar-Ilan, J., Levene, M. et al. Comparing university rankings. Scientometrics 85, 243–256 (2010). https://doi.org/10.1007/s11192-010-0190-z

Download citation

Keywords

  • Ranking
  • Universities
  • Shanghai ranking
  • Times ranking
  • Taiwan ranking
  • Leiden ranking
  • Webometrics ranking
  • Comparative analysis