Skip to main content
SpringerLink
Log in

Uncovering excellence in academic rankings: a closer look at the Shanghai ranking

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

In this paper, we examine whether the quality of academic research can be accurately captured by a single aggregated measure such as a ranking. With Shanghai University’s Academic Ranking of World Universities as the basis for our study, we use robust principal component analysis to uncover the underlying factors measured by this ranking. Based on a sample containing the top 150 ranked universities, we find evidence that, for the majority of these institutions, the Shanghai rankings reflect not one but in fact two different and uncorrelated aspects of academic research: overall research output and top-notch researchers. Consequently, the relative weight placed upon these two factors determines to a large extent the final ranking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. See http://www.ed.sjtu.edu.cn/ranking.htm.

  2. In standardized PCA, the eigenvectors are obtained from the correlation matrix.

  3. Computed after removing the variable “PCP” and transferring its weight equally between the remaining five variables.

  4. Since we are working here with standardized variables, the overall variance here is equal to five, the number of variables.

  5. Obviously, if each university were to contribute an equal share in a factor, that contribution would be 0.67%.

  6. The first principal component is measured on the first principal axis.

  7. Under the normality assumption, the Mahalanobis distances are distributed as the square root of a χ2 distribution with five degrees of freedom.

  8. The STATA code is available from the authors upon request (Verardi and Dehon 2009).

References

  • Dehon, C., Jacobs, D., & Vermandele, C. (2009). Ranking Universities. Brussels: Editions de l’Université de Bruxelles.

    Google Scholar 

  • Liu N. C., & Cheng, Y. (2005). Academic ranking of world universities: Methodologies and problems. Higher Education in Europe, 30(2), 127–136.

    Google Scholar 

  • Mahalanobis, P. C. (1936). On the generalized distance in statistics. Proceedings of the National Institute of Sciences of India, Calcutta, 2(1), 49–55.

    MATH  Google Scholar 

  • Marcotte, C., Desroches, J., & Poupart I. (2007). Preparing internationally minded business graduates: The role of international mobility programs. International Journal of Intercultural Relations, 31, 655–668.

    Article  Google Scholar 

  • Razvan, F. (2007). Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics, 72(1), 25–32.

    Article  Google Scholar 

  • Rousseeuw P. J. (1985). Multivariate estimation with high breakdown point. Mathematical Statistics and Applications, B, 283–297.

    MathSciNet  Google Scholar 

  • van Raan, A. (2005a). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.

    Article  Google Scholar 

  • van Raan, A. (2005b). Challenges in ranking of Universities, Invited paper for the first international conference on World Class Universities. Shanghai: Shanghai Jaio Tong University, June 16–18, 2005.

  • Verardi, V., & Dehon C. (2009). Multivariate outlier detection in stata. Working paper.

  • Vincke, Ph. (2009). University rankings. In C. Dehon, D. Jacobs, & C. Vermandele (Eds.), Ranking Universities (pp. 11–25). Brussels: Editions de l’Université de Bruxelles.

  • Zitt, M., & Filliatreau, G. (2007). Big is (made) beautiful: Some comments about the Shanghai ranking of world-class universities. In J. Sadlack, & N. C. Liu (Eds.), The world class university and ranking: Aiming beyond status, (pp. 141–160). Romania: UNESCO-CEPES, Cluj University Press.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Nadine Rons, Ellen Hazelkorn and Marjorie Gassner for their insightful comments. Vincenzo Verardi is Associate Researcher of the FNRS and gratefully acknowledges their financial support. All remaining errors are the authors' responsibility.

Author information

Authors and Affiliations

  1. Université Libre de Bruxelles, Brussels, Belgium

    Catherine Dehon, Alice McCathie & Vincenzo Verardi

  2. University of Namur, Namur, Belgium

    Vincenzo Verardi

Authors
  1. Catherine Dehon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alice McCathie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vincenzo Verardi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Catherine Dehon.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dehon, C., McCathie, A. & Verardi, V. Uncovering excellence in academic rankings: a closer look at the Shanghai ranking. Scientometrics 83, 515–524 (2010). https://doi.org/10.1007/s11192-009-0076-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-009-0076-0

Keywords

Search

Navigation