Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A fresh approach to evaluating the academic ranking of world universities

Abstract

The aim of this article is to present new ideas in evaluating Shanghai University’s Academic Ranking of World Universities (ARWU). One issue frequently put forth in various publications is that the Shanghai rankings are sensitive to the relative weight they attribute to each variable. As a possible remedy to this issue, the statistical I-distance method is proposed to be used. Based on a sample containing the top 100 ranked universities, the results show a significant correlation with the official ARWU list. However, some inconsistencies concerning European universities have been noticed and elaborated upon.

This is a preview of subscription content, log in to check access.

References

  1. Agasisti, T., & Perez-Esparrells, C. (2010). Comparing efficiency in a cross-country perspective: The case of Italian and Spanish state universities. Higher Education, 59, 85–103.

  2. Aguillo, I., Bar-IIan, J., Levene, M., & Ortega, J. L. (2010). Comparing university rankings. Scientometrics, 85(1), 243–256.

  3. Billaut, J. C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shanghai ranking: An MCDM view. Scientometrics, 84(1), 237–263.

  4. Bowman, N., & Bastedo, M. (2010). Anchoring effects in world university rankings: Exploring biases in reputation scores. Higher Education. doi: 10.1007/s10734-010-9339-1.

  5. Dehon, C., McCathie, A., & Verardi, V. (2010). Uncovering excellence in academic rankings: A closer look at the Shanghai ranking. Scientometrics, 83(2), 515–524.

  6. Docampo, D. (2008). International rankings and quality of the university systems. Revista Education, 15(SI), 149–176.

  7. Docampo, D. (2011). On using the Shanghai ranking to assess the research performance of university systems. Scientometrics, 86(1), 77–92.

  8. Florian, R. V. (2007). Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics, 72(1), 25–32.

  9. Hien, P. D. (2010). A comparative study of research capabilities of East Asian countries and implications for Vietnam. Higher Education, 60, 615–625.

  10. Ivanovic, B. (1973). A method of establishing a list of development indicators. Paris: United Nations educational, scientific and cultural organization.

  11. Ivanovic, B. (1977). Classification theory. Belgrade: Institute for Industrial Economics.

  12. Ivanovic, B., & Fanchette, S. (1973). Grouping and ranking of 30 countries of Sub-Saharan Africa, two distance-based methods compared. Paris: United Nations educational, scientific and cultural organization.

  13. Jeremic, V., & Radojicic, Z. (2010). A new approach in the evaluation of team chess championships rankings. Journal of Quantitative Analysis in Sports, 6(3). doi:10.2202/1559-0410.1257.

  14. Liu, N. C., & Cheng, Y. (2005). Academic ranking of world universities: Methodologies and problems. Higher Education in Europe, 30(2), 127–136.

  15. Liu, N. C., Cheng, Y., & Liu, L. (2005). Academic ranking of world universities using scientometrics: A comment to the “fatal attraction”. Scientometrics, 64(1), 101–109.

  16. Lukman, R., Krajnc, D., & Glavic, P. (2010). University ranking using research, educational and environmental indicators. Journal of Cleaner Production, 18(7), 619–628.

  17. Mihailovic, N., Bulajic, M., & Savic, G. (2009). Ranking of banks in Serbia. Yugoslav Journal of Operations Research, 19(2), 323–334. doi:10.2298/YUJOR0902323M.

  18. Stolz, I., Hendel, D., & Horn, A. (2010). Ranking of rankings: Benchmarking twenty-five higher education ranking systems in Europe. Higher Education, 60(5), 507–528.

  19. van Raan, A. F. J. (2005a). Fatal attraction: Ranking of universities by bibliometric methods. Scientometrics, 62, 133–145.

  20. van Raan, A. F. J. (2005b). Reply to the comments of Liu et al. Scientometrics, 64(1), 111–112.

Download references

Author information

Correspondence to Veljko Jeremic.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Jeremic, V., Bulajic, M., Martic, M. et al. A fresh approach to evaluating the academic ranking of world universities. Scientometrics 87, 587–596 (2011). https://doi.org/10.1007/s11192-011-0361-6

Download citation

Keywords

  • Ranking of universities
  • The I-distance method
  • ARWU
  • Statistical methods
  • Classification