Skip to main content
SpringerLink
Log in

Inter-ranking reputational effects: an analysis of the Academic Ranking of World Universities (ARWU) and the Times Higher Education World University Rankings (THE) reputational relationship

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Global university rankings have become a critical factor in the higher education sector, engendering increasing interest and exerting a notable influence over a wide variety of stakeholders. They are presented to different audiences as tools that evaluate and rank universities according to quality. However, some authors are of the opinion that rankings express reputational factors to a large extent. This article presents a model of the intra- and inter-ranking relationships from the perspective of reputation along with an empirical study on two of the most influential rankings: the Academic Ranking of World Universities and the Times Higher Education World University Rankings. Data from these two rankings between 2010 and 2018, and the application of ordinal regressions, provide evidence that both rankings are mutually influential, generating intra and reciprocal reputational effects over time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. See Van Raan (2005) for a discussion on the major problems in the application of publication and citation data in the context of evaluation.

  2. This statement refers to the communication of the results of rankings, which complement other types of communication that also influence perceptions, e.g. speeches given by vice-chancellors or rectors (Rodriguez-Pomeday and Casani 2016) or the degree of news coverage of the university (Arpan et al. 2003), among others.

  3. This data is provided by universities themselves and is easily manipulated (Waltman et al. 2012), which explains the high fluctuation from year to year. Rankings based entirely on bibliometric data do not suffer from this type of problem.

  4. According to the THE methodology “Universities can be excluded from the World University Rankings if they do not teach undergraduates, or if their research output amounted to fewer than 1,000 relevant publications between 2013 and 2017 (with a minimum of 150 a year). Universities can also be excluded if 80 per cent or more of their research output is exclusively in one of our 11 subject areas” (timeshighereducation.com). The complete inclusion criteria appear in the document Methodology for overall and subject rankings for the Times Higher Education World University Rankings 2019 (September 2018), published in https://www.timeshighereducation.com/sites/default/files/the_2019_world_university_rankings_methodology_pwc.pdf

  5. The THE ranking was preferred over QS due to its wider audience, but also because of the greater overlap with ARWU. Shehatta and Mahmood (2016) observe that ARWU and QS 2015 results are the lowest overlapping in terms of universities among the six global rankings studied.

  6. Overall rankings are taken. These rankings are those that are disseminated by the media and have reputational effects.

References

  • Aguillo, I., Bar-Ilan, J., Levene, M., & Ortega, J. (2010). Comparing university rankings. Scientometrics, 85(1), 243–256.

    Article  Google Scholar 

  • Aldrich, J., & Nelson, F. (1984). Linear probability, logit and probit models. Sage university paper series on quantitative applications in the social sciences. Beverly Hills: Sage.

    Book  Google Scholar 

  • Argenti, P. (2000). Branding b-schools: Reputation management for MBA programs. Corporate Reputation Review, 3(2), 171–178.

    Article  Google Scholar 

  • Armstrong, J. S., & Sperry, T. (1994). Business school prestige-research versus teaching. Interfaces, 24(2), 13–43.

    Article  Google Scholar 

  • Arpan, L. M., Raney, A. A., & Zivnuska, S. (2003). A cognitive approach to understanding university image. Corporate Communications: An International Journal, 8(2), 97–113.

    Article  Google Scholar 

  • ARWU. (2018). About ARWU. http://www.shanghairanking.com/aboutarwu.html Accessed 19 December 2018.

  • Baghestanian, S., & Popov, S. V. (2014). Alma matters: Determinants of early career success in economics. Technical report, Goethe University.

  • Bastedo, M. N., & Bowman, N. A. (2009). US news & World report college rankings: Modeling institutional effects on organizational reputation. American Journal of Education, 116(2), 163–183.

    Article  Google Scholar 

  • Bentley, R., & Blackburn, R. (1990). Changes in academic research performance over time: A study of institutional accumulative advantage. Research in Higher Education, 31(4), 327–353.

    Article  Google Scholar 

  • Bhattacharjee, Y. (2011). Saudi universities offer cash in exchange for academic prestige. Science, 334, 1344–1345.

    Article  Google Scholar 

  • Blank, R. M. (1991). The effects of double-blind versus single-blind reviewing: Evidence from the American economic review. American Economic Review, 81, 1041–1067.

    Google Scholar 

  • Bolbanabad, A. M., Mosadeghrad, A. M., Arab, M., & Majdzadeh, R. (2017). Impact of merger and acquisition on university performance. Archives of Iranian Medicine, 20(8), 518–524.

    Google Scholar 

  • Bookstein, F. L., Seidler, H., Fieder, M., & Winckler, G. (2010). Too much noise in the times higher education rankings. Scientometrics, 85(1), 295–299.

    Article  Google Scholar 

  • Bowman, N. A., & Bastedo, M. N. (2011). Anchoring effects in world university rankings: Exploring biases in reputation scores. High Education, 61, 431–444.

    Article  Google Scholar 

  • Brooks, R. (2005). Measuring university quality. The Review of Higher Education, 29(1), 1–21.

    Article  MathSciNet  Google Scholar 

  • Çakır, M. P., Acartürk, C., Alaşehir, O., & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813–848.

    Article  Google Scholar 

  • Clark, C., & Wright, M. (2007). Reviewing journal rankings and revisiting peer reviews: Editorial perspectives. Journal of Management Studies, 44(4), 612–621.

    Article  Google Scholar 

  • Cole, S., & Cole, J. R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American Sociological Review, 32(3), 377–390.

    Article  Google Scholar 

  • Dey, E. L., Milem, J. F., & Berger, J. B. (1997). Changing patterns of publication productivity: Accumulative advantage or institutional isomorphism? Sociology of Education, 70(4), 308–323.

    Article  Google Scholar 

  • Dichev, I. (1999). How good are business school’s rankings? Journal of Business, 72(2), 201–213.

    Article  Google Scholar 

  • Dichev, I. (2001). News or noise? Research in Higher Education, 42(3), 237–266.

    Article  Google Scholar 

  • Docampo, D., Egret, D., & Cram, L. (2015). The effect of university mergers on the Shanghai ranking. Scientometrics, 104(1), 175–191.

    Article  Google Scholar 

  • Dyck, I. & Zingales, L. (2002). The corporate governance role of the media. CRSP working paper, no. 543. Retrieved October 19, 2005. from ssrn.com/abstract = 335602.

  • Falagas, M. E., Zouglakis, G. M., & Kavvadia, P. K. (2006). How masked is the “masked peer review” of abstracts submitted to international medical conferences? Mayo Clinic Proceedings, 81(5), 705.

    Article  Google Scholar 

  • Fombrun, C. (1996). Reputation: Realizing value from the corporate image. Boston: Harvard Business School Press.

    Google Scholar 

  • Fowles, J., Frederickson, H. G., & Koppell, J. G. (2016). University rankings: Evidence and a conceptual framework. Public Administration Review, 76(5), 790–803.

    Article  Google Scholar 

  • Garfunkel, J. M., Ulshen, M. H., Hamrick, H. J., & Lawson, E. E. (1994). Effect of institutional prestige on reviewers’ recommendations and editorial decisions. JAMA, 272(2), 137–138.

    Article  Google Scholar 

  • Gnolek, S. L., Falciano, V. T., & Kuncl, R. W. (2014). Modeling change and variation in US news & World report college rankings: What would it really take to be in the top 20? Research in Higher Education, 55(8), 761–779.

    Article  Google Scholar 

  • González-Riaño, G., Repiso Caballero, R., & Delgado López-Cózar, E. (2014). The impact of university rankings in the Spanish press (Repercusión de los rankings universitarios en la prensa española). Revista Española de Documentación Científica, 37(3), e055.

    Article  Google Scholar 

  • Harman, G., & Harman, K. (2008). Strategic mergers of strong institutions to enhance competitive advantage. Higher Education Policy, 21(1), 99–121.

    Article  Google Scholar 

  • Hazelkorn, E. (Ed.). (2016). Global rankings and the geopolitics of higher education: Understanding the influence and impact of rankings on higher education, policy and society. Abingdon: Taylor & Francis.

    Google Scholar 

  • Hazelkorn, E. (2017). Rankings and higher education: Reframing relationships within and between states. Centre for Global Higher Education working paper no. WC1H 0AL. London, the UK: UCL Institute of Education.

  • Judge, T. A., Cable, D. M., Colbert, A. E., & Rynes, S. L. (2007). What causes a management article to be cited—Article, author, or journal? Academy of Management Journal, 50(3), 491–506.

    Article  Google Scholar 

  • Kane, K. F. (1993). MBAs: A recruiter’s-eye view: Important criteria for MBA positions. Business Horizons, 36(1), 65–71.

    Article  Google Scholar 

  • Kauppi, N. (2018). The global ranking game: Narrowing academic excellence through numerical objectification. Studies in Higher Education, 43(10), 1750–1762.

    Article  Google Scholar 

  • Keith, B. (2001). Organizational contexts and university performance outcomes: The limited role of purposive action in the management of institutional status. Research in Higher Education, 42(5), 493–516.

    Article  Google Scholar 

  • Keller, K., Apéria, T., & Georgson, M. (2008). Strategic brand management. Upper Saddle River: Prentice Hall.

    Google Scholar 

  • Kohm, L. M., & Wardle, L. D. (2011). The echo-chamber effect in legal education: Considering family law casebooks. St. Thomas Journal of Law & Public Policy, 6(1), 104–140.

    Google Scholar 

  • Larivière, V., & Gingras, Y. (2010). The impact factor’s matthew effect: A natural experiment in bibliometrics. Journal of the American Society for Information Science and Technology, 61(2), 424–427.

    Google Scholar 

  • Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.

    Article  Google Scholar 

  • Leiter, B. (2006). Commentary: How to rank law schools. Indiana Law Journal, 81, 47–52.

    Google Scholar 

  • Locke, W., Verbik, L., Richardson, J., & King, R. (2008). Counting what is measured or measuring what counts? League tables and their impact on higher education institutions in England. Bristol: Higher Education Funding Council for England.

    Google Scholar 

  • Lou, W., & He, J. (2015). Does author affiliation reputation affect uncitedness? In Proceedings of the 78th ASIS&T annual meeting: Information science with impact: Research in and for the community (p. 103). American Society for Information Science.

  • Luca, M., & Smith, J. (2015). Strategic disclosure: The case of business school rankings. Journal of Economic Behavior & Organization, 112, 17–25.

    Article  Google Scholar 

  • McGillivray, B., & De Ranieri, E. (2018). Uptake and outcome of manuscripts in Nature journals by review model and author characteristics. Research Integrity and Peer Review, 3(5), 1–12.

    Google Scholar 

  • Medoff, M. H. (2006). Evidence of a harvard and Chicago Matthew effect. Journal of Economic Methodology, 13(4), 485–506.

    Article  Google Scholar 

  • Merton, R. K. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63.

    Article  Google Scholar 

  • Messerly, M. (2014). Citations for sale: Saudi university boosts rankings by aggressively recruiting world’s top researchers. The daily Californian. http://www.dailycal.org/2014/12/05/citations-sale. Retrieve 1 December 2018.

  • Meyer, M., Waldkirch, R. W., Duscher, I., & Just, A. (2018). Drivers of citations: An analysis of publications in “top” accounting journals. Critical Perspectives on Accounting, 51, 24–46.

    Article  Google Scholar 

  • Miller, C. C. (2006). Peer review in the organizational and management sciences: Prevalence and effects of reviewer hostility, bias, and dissensus. Academy of Management Journal, 49(3), 425–431.

    Article  Google Scholar 

  • Moed, H. F. (2017). A critical comparative analysis of five world university rankings. Scientometrics, 110(2), 967–990.

    Article  Google Scholar 

  • Moed, H. F., Burger, W. J. M., Frankfort, J. G., & Van Raan, A. F. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14(3), 131–149.

    Article  Google Scholar 

  • Morgeson, F. P., & Nahrgang, J. D. (2008). Same as it ever was: Recognizing stability in the business week rankings. Academy of Management Learning & Education, 7, 26–41.

    Article  Google Scholar 

  • Olcay, G. A., & Bulu, M. (2017). Is measuring the knowledge creation of universities possible?: A review of university rankings. Technological Forecasting and Social Change, 123, 153–160.

    Article  Google Scholar 

  • Oswald, A. J. (2008). Can we test for bias in scientific peer-review? IZA DP No. 3665, Forschungsinstitut zur Zukunft der Arbeit Institute for the Study of Labor.

  • Pang, L. (2018). How Tsinghua became a world class research university: A case study on the impact of rankings on a Chinese higher education institution (doctoral dissertation).

  • Peters, D. P., & Ceci, S. J. (1982). A naturalistic study of psychology journals: The fate of published articles resubmitted. Behavioral and Brain Sciences, 5, 187–252.

    Article  Google Scholar 

  • Pinar, M., Milla, J., & Stengos, T. (2019). Sensitivity of university rankings: implications of stochastic dominance efficiency analysis. Education Economics, 27(1), 75–92.

    Article  Google Scholar 

  • Piro, F. N., & Sivertsen, G. (2016). How can differences in international university rankings be explained? Scientometrics, 109(3), 2263–2278.

    Article  Google Scholar 

  • Rauhvargers, A. (2011). Global university rankings and their impact. Brussels: European University Association.

    Google Scholar 

  • Rindova, V. P., Williamson, I. O., Petkova, A. P., & Sever, J. M. (2005). Being good or being known: An empirical examination of the dimensions, antecedents, and consequences of organizational reputation. Academy of Management Journal, 48(6), 1033–1049.

    Article  Google Scholar 

  • Rodriguez-Pomeday, J., & Casani, F. (2016). Legitimating the world-class university concept through the discourse of elite universities’ presidents. Higher Education Research and Development, 35(6), 1269–1283.

    Article  Google Scholar 

  • Rondeau, K. V. (2017). The impact of world ranking systems on graduate schools of business: Promoting the manipulation of image over the management of substance. World Journal of Education, 7(3), 62–73.

    Article  Google Scholar 

  • Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., Grant, A. O., Daniels, S. R., et al. (2006). Effect of blinded peer review on abstract acceptance. JAMA, 295(14), 1675–1680.

    Article  Google Scholar 

  • Safón, V. (2007). Factors that influence recruiters’ choice of business schools and their MBA graduates: Evidence and implications for business schools. Academy of Management Learning and Education, 6(2), 217–233.

    Article  Google Scholar 

  • Safón, V. (2009). Measuring the reputation of top US business schools: A MIMIC modeling approach. Corporate Reputation Review, 12(3), 204–228.

    Article  Google Scholar 

  • Safón, V. (2012). Can the reputation of an established business school change? Management in Education, 26(4), 169–180.

    Article  Google Scholar 

  • Safón, V. (2013). What do global university rankings really measure? The search for the X factor and the X entity. Scientometrics, 97(2), 223–244.

    Article  Google Scholar 

  • Saisana, M., d’Hombres, B., & Saltelli, A. (2011). Rickety numbers: Volatility of university rankings and policy implications. Research Policy, 40(1), 165–177.

    Article  Google Scholar 

  • Shehatta, I., & Mahmood, K. (2016). Correlation among top 100 universities in the major six global rankings: Policy implications. Scientometrics, 109(2), 1231–1254.

    Article  Google Scholar 

  • Shin, J. C. (2011). Chapter 2. Organizational effectiveness and university rankings. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings the changing academy—The changing academic profession in international comparative perspective 3. Berlin: Springer.

    Google Scholar 

  • Smyth, D. J. (1999). The determinants of the reputations of economics departments: Pages published, citations and the Andy Rooney effect. American Economist, 43(2), 49–58.

    Article  Google Scholar 

  • Sorz, J., Wallner, B., Seidler, H., & Fieder, M. (2015). Inconsistent year-to-year fluctuations limit the conclusiveness of global higher education rankings for university management. PeerJ, 3(e1217), 1–14.

    Google Scholar 

  • Spence, M. (1973). Job market signaling. Quarterly Journal of Economics, 87(3), 355–374.

    Article  Google Scholar 

  • Stake, J. E. (2006). The interplay between law school rankings, reputations, and resource allocation: Ways rankings mislead. Indiana Law Journal, 81, 229–270.

    Google Scholar 

  • Stella, A. & Woodhouse, D. (2006). Ranking of higher education institutions. AUQA Occasional Publications Number 6. Australian Universities Quality Agency.

  • Sweitzer, K., & Volkwein, F. (2009). Prestige among graduate and professional schools: Comparing the U. S. news’ graduate school reputation ratings between disciplines. Research in Higher Education, 50(8), 812–836.

    Article  Google Scholar 

  • Toutkoushian, R. K., & Webber, K. (2011). Chapter 7. Measuring the research performance of postsecondary institutions. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings, the changing academy—The changing academic profession in international comparative perspective 3. Berlin: Springer.

    Google Scholar 

  • Van Raan, A. F. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.

    Article  Google Scholar 

  • Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E., Tijssen, R., Eck, N., et al. (2012). The Leiden ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology, 63(12), 2419–2432.

    Article  Google Scholar 

  • Wang, X., Liu, C., Mao, W., & Fang, Z. (2015). The open access advantage considering citation, article usage and social media attention. Scientometrics, 103(2), 555–564.

    Article  Google Scholar 

  • Wedlin, L. (2007). The role of rankings in codifying a business school template: Classifications, diffusion and mediated isomorphism in organizational fields. European Management Review, 4(1), 24–39.

    Article  Google Scholar 

  • Yotopoulos, P. A. (1961). Institutional affiliation of the contributors to three professional journals. American Economic Review, 51, 665–670.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management, Faculty of Economics, University of Valencia, Avenida de los Naranjos, s/n, 46022, Valencia, Spain

    Vicente Safón

Authors
  1. Vicente Safón
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vicente Safón.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Safón, V. Inter-ranking reputational effects: an analysis of the Academic Ranking of World Universities (ARWU) and the Times Higher Education World University Rankings (THE) reputational relationship. Scientometrics 121, 897–915 (2019). https://doi.org/10.1007/s11192-019-03214-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-019-03214-9

Keywords

Mathematical Subject Classification

JEL Classification

Search

Navigation