Abstract
Despite some theoretical and technical criticism, scholars largely acknowledge the influence of universities’ ranking positions on the preferences of fund providers, academics and students, nationally and internationally. Considering their noticeable contribution to university rankings, prominent indicators can guide university leaders to develop better strategies by targeting common aspects of international ranking systems. The purpose of this research is therefore to specify the significant indicators and to examine their individual weight through an expanded indicator-set of international university rankings. The research benefited from the predictive approach of correlational research. The dataset was composed of universities’ scores in the 2018 ARWU, THE, QS and URAP world university rankings and includes the scores of 224 universities. The data were re-organised following the expanded indicator-set previously formulated by the researcher. Regression analyses were then employed in two steps to explore significant predictors through the expanded indicator-set. The researcher also re-calculated the percentage values of seven combined indicators: citation, income, internationalisation, prize, publication, reputation and ratios/degrees. The findings showed that while all these indicators are statistically significant, the components of research reputation contribute 73.71% to universities’ ranking scores. On the other hand, income is the only negative contributor with a weight of − 1.78%. The research also revealed that when comparing two scores based on re-calculated and assigned weights, only 19 universities occupy the same position among the 224 universities. Following these results, the researcher then discusses various policies and practices with the potential to expedite universities’ ranking success. Considering the data reliability and longitudinal feasibility, several recommendations were also developed for further research on university ranking systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In predictive analyses, beta coefficient (β) indicates the individual contribution of the independent variable to the dependent variable. Here, for example, β = 662 for ARWU means a 1-point increase in ARWU brought a .662-point increase in the combination of ARWU, QS and THE.
In the predictive analysis, the determinant coefficient (R2) indicates the explanation ratio of dependent (predicted) variables by independent (predictive) variables. Then, R2 = 1 means all predictor variables that together explain the whole constitution of the predicted variable. In this research, while the un-standardised beta (β) coefficient of each indicator shows its assigned value by the related ranking system (ARWU, QS, THE or URAP), the standardised β coefficient reveals the individual contribution of the related indicator (see previous example in Endnote-1 above).
While the seven indicators together fully explain universities’ ranking scores (according to R2 = .997), standardised β values show “Ranking Score = .313x(publication) + .285x(reputation) + .274x(citation) + .150x(ratios/degrees) + .104x(prizes) + .078x(internationalisation)−.021x(income)”. t and p values together also indicate valid test results for individual contribution of the related indicator at significance level (here .05)
e.g. Percentage value of publication = [.313/(.313 + .285 + .274 + .150 + .104 + .078–.021)] × 100 = 26.46%.
e.g. Percentage value of ARWU-Publication = [.266 / (.266 + .265 + .231 + .172 + .099 + .045)] × 26.46 = 6.53%.
e.g. 32.75% in total 400% of ARWU, QS, THE and URAP (see Table 1); 32.75%/4 = 8.19%; 12.68–8.19% = 4.49%
e.g. 20% in total 400% of ARWU, QS, THE and URAP (see Table 1); 20%/4 = 5%; 9.26–5.00% = 4.26%
e.g. ([Number of Patent applications/100] × 10) + ([Granted Patent Percentage/100] × 10) + ([Commercial Impact Score/100] × 80)
Re-calculation by the above formula resulted in different ranks for 95 universities, and the top university lost ground to sixth position.
References
5top100. (n.d.). Russian Academic Excellence Project: World-class Russian education. Accessed (July 23, 2019): https://www.5top100.ru/en/about/more-about/
Abramo, G., D’Angelo, C. A., & Solazzi, M. (2011). The relationship between scientists’ research performance and the degree of internationalization of their research. Scientometrics, 86(3), 629–643. https://doi.org/10.1007/s11192-010-0284-7.
Alden, J., & Lin, G. (2004). Benchmarking the characteristics of a world-class university: Developing an international strategy at university level. London: Leadership Foundation for Higher Education.
Altbach, P. G., & Salmi, J. (2011). The road to academic excellence: The making of world-class research universities. Washington, DC: The World Bank.
ARWU. (2018a). ShanghaiRanking’s Academic Ranking of World Universities 2018 Press Release. Accessed (August 15, 2018): http://www.shanghairanking.com/Academic-Ranking-of-World-Universities-2018-Press-Release.html.
ARWU. (2018b). Methodology, Academic Ranking of World Universities. Accessed (August 16, 2018): http://www.shanghairanking.com/ARWU-Methodology-2018.html.
Ary, D., Jacobs, L. C., Razavieh, A., & Sorensen, C. (2006). Introduction to research in education (7th ed.). Belmont, CA: Thomson Wadsworth.
Biesta, G. (2009). Good education in an age of measurement: on the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability, 21(1), 33–46. https://doi.org/10.1007/s11092-008-9064-9.
Bougnol, M.-L., & Dula, J. H. (2015). Technical pitfalls in university rankings. Higher Education, 69(5), 859–866. https://doi.org/10.1007/s10734-014-9809-y.
Bowman, N. A., & Bastedo, M. N. (2011). Anchoring effects in world university rankings: Exploring biases in reputation scores. Higher Education, 61(4), 431–444. https://doi.org/10.1007/s10734-010-9339-1.
Cadez, S., Dimovski, V., & Groff, M. Z. (2017). Research, teaching and performance evaluation in academia: The salience of quality. Studies in Higher Education, 42(8), 1455–1473. https://doi.org/10.1080/03075079.2015.1104659.
Çakır, M. P., Acartürk, C., Alaşehir, O., & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813–848. https://doi.org/10.1007/s11192-015-1586-6.
Chapleo, C. (2010). What defines “successful” university brands? International Journal of Public Sector Management, 23(2), 169–183. https://doi.org/10.1108/09513551011022519.
Collins, F. L., & Park, G.-S. (2016). Ranking and the multiplication of reputation: reflections from the frontier of globalizing higher education. Higher Education, 72(1), 115–129. https://doi.org/10.1007/s10734-015-9941-3.
Delgado-Marquez, B. L., Escudero-Torres, M. A., & Hurtado-Torres, N. E. (2013). Being highly internationalised strengthens your reputation: an empirical investigation of top higher education institutions. Higher Education, 66(5), 619–633. https://doi.org/10.1007/s10734-013-9626-8.
DFG. (n.d.). Excellence Initiative (2005–2017/19). Accessed (July 23, 2019): https://www.dfg.de/en/research_funding/programmes/excellence_initiative/index.html.
Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: a cross-national analysis of university ranking systems. Higher Education, 49(4), 495–533. https://doi.org/10.1007/s10734-004-1746-8.
Gargouri, Y., Hajjem, C., Lariviere, V., Gingras, Y., Carr, L., Brody, T., & Harnad, S. (2010). Self-selected or mandated, open access increases citation impact for higher quality research. PLoS One, 5(10), e13636. https://doi.org/10.1371/journal.pone.0013636.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Pearson Prentice Hall.
Harvey, L. (2008). Rankings of higher education institutions: a critical review. Quality in Higher Education, 14(3), 187–207. https://doi.org/10.1080/13538320802507711.
Hazelkorn, E. (2015). Rankings and the reshaping of higher education: The battle for world-class excellence (2nd ed.). London: Palgrave Macmillan.
Heffernan, T. A., & Heffernan, A. (2018). Language games: university responses to ranking metrics. Higher Education Quarterly, 72(1), 29–39. https://doi.org/10.1111/hequ.12139.
Hendel, D. D., & Stolz, I. (2008). A comparative analysis of higher education ranking systems in Europe. Tertiary Education and Management, 14(3), 173–189. https://doi.org/10.1080/13583880802228158.
Horta, H., & Santos, J. M. (2016). The impact of publishing during PhD studies on career research publication, visibility, and collaborations. Research in Higher Education, 57(1), 28–50. https://doi.org/10.1007/s11162-015-9380-0.
Jajo, N. K., & Harrison, J. (2014). World university ranking systems: an alternative approach using partial least squares path modelling. Journal of Higher Education Policy and Management, 36(5), 471–482. https://doi.org/10.1080/1360080X.2014.936090.
Jarwal, S. D., Brion, A. M., & King, M. L. (2009). Measuring research quality using the journal impact factor, citations and ‘ranked journals’: blunt instruments or inspired metrics? Journal of Higher Education Policy and Management, 31(4), 289–300. https://doi.org/10.1080/13600800903191930.
Kwiek, M. (2015). The internationalization of research in Europe: a quantitative study of 11 national systems from a micro-level perspective. Journal of Studies in International Education, 19(4), 341–359. https://doi.org/10.1177/1028315315572898.
Lim, M. A. (2018). The building of weak expertise: the work of global university rankers. Higher Education, 75(3), 415–430. https://doi.org/10.1007/s10734-017-0147-8.
Liu, Z., Moshi, G. L., & Awuor, C. M. (2019). Sustainability and indicators of newly formed world-class universities (NFWCUs) between 2010 and 2018: empirical analysis from the rankings of ARWU, QSWUR and THEWUR. Sustainability, 11(10), 1–29. https://doi.org/10.3390/su11102745.
Luque-Martínez, T., & Faraoni, N. (2019). Meta-ranking to position world universities. Studies in Higher Education (Online First: 1-15). https://doi.org/10.1080/03075079.2018.1564260.
Macfarlane, B. (2007). Defining and rewarding academic citizenship: the implications for university promotions policy. Journal of Higher Education Policy and Management, 29(3), 261–273. https://doi.org/10.1080/13600800701457863.
McGrail, M. R., Rickard, C. M., & Jones, R. (2006). Publish or perish: a systematic review of interventions to increase academic publication rates. Higher Education Research & Development, 25(1), 19–35. https://doi.org/10.1080/07294360500453053.
PREC Edu Service. (n.d.). China’s New World-Class University Policy. Accessed (July 22, 2019): https://www.studyinchina.com.my/web/page/chinas-new-world-class-university-policy/.
QS. (2018). Methodology, QS World University Rankings. Accessed (January 13, 2018): https://www.topuniversities.com/qs-world-university-rankings/methodology.
Reuters. (2018a). Methodology: Top 100 Innovative Universities – 2018. Accessed (July 13, 2019): https://www.reuters.com/innovative-universities-2018/methodology.
Reuters. (2018b). Reuters Top 100: The World's Most Innovative Universities - 2018. Accessed (July 13, 2019): https://www.reuters.com/article/us-amers-reuters-ranking-innovative-univ/reuters-top-100-the-worlds-most-innovative-universities-2018-idUSKCN1ML0AZ.
Salmi, J. (2009). The challenge of establishing world-class universities. Washington, DC: The World Bank.
Shattock, M. (2017). The ‘world class’ university and international ranking systems: What are the policy implications for governments and institutions? Policy Reviews in Higher Education, 1(1), 4–21. https://doi.org/10.1080/23322969.2016.1236669.
Shin, J. C. (2013). The world-class university: Concept and policy initiatives. In J. C. Shin & B. M. Kehm (Eds.), Institutionalization of world-class university in global competition (pp. 17–32). Dordrecht: Springer.
Shin, J. C., & Toutkoushian, R. K. (2011). The past, present, and future of university rankings. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (pp. 1–16). Dordrecht: Springer.
Shin, J. C., Toutkoushian, R. K., & Teichler, U. (2011). University rankings: Theoretical basis, methodology and impacts on global higher education. Dordrecht: Springer.
Soh, K. (2015). What the overall doesn’t tell about world university rankings: examples from ARWU, QSWUR, and THEWUR in 2013. Journal of Higher Education Policy and Management, 37(3), 295–307. https://doi.org/10.1080/1360080X.2015.1035523.
Soh, K. (2017). The seven deadly sins of world university ranking: a summary from several papers. Journal of Higher Education Policy and Management, 39(1), 104–115. https://doi.org/10.1080/1360080X.2016.1254431.
Spence, C. (2019). ‘Judgement’ versus ‘metrics’ in higher education management. Higher Education, 77(5), 761–775. https://doi.org/10.1007/s10734-018-0300-z.
Tapper, T., & Filippakou, O. (2009). The world-class league tables and the sustaining of international reputations in higher education. Journal of Higher Education Policy and Management, 31(1), 55–66. https://doi.org/10.1080/13600800802383091.
Tatsioni, A., Vavva, E., & Ioannidis, J. P. A. (2010). Sources of funding for Nobel Prize-winning work: public or private? The FASEB Journal, 24(5), 1335–1339. https://doi.org/10.1096/fj.09-148239.
Tekbıyık, A. (2014). İlişkisel tarama yöntemi [Correlational survey model]. In M. Metin (Ed.), Kuramdan uygulamaya eğitimde bilimsel araştırma yöntemleri [from theory to practices, research methods in education] (pp. 99–114). Ankara: Pegem.
THE. (2018a). World University Rankings 2018 Methodology. Accessed (January 14, 2018): https://www.timeshighereducation.com/world-university-rankings/methodology-world-university-rankings-2018.
THE. (2018b). THE World Reputation Rankings 2018: Methodology explained. Accessed (July 13, 2019): https://www.timeshighereducation.com/world-university-rankings/world-reputation-rankings-2018-methodology.
THE. (2019a). World University Rankings 2019. Accessed (July 13, 2019): https://www.timeshighereducation.com/world-university-rankings/2019/world-ranking#!/page/0/length/25/sort_by/rank/sort_order/asc/cols/stats.
THE. (2019b). THE University Impact Rankings 2019: Methodology. Accessed (July 13, 2019): https://www.timeshighereducation.com/world-university-rankings/methodology-impact-rankings-2019.
TÜBİTAK. (2019). 2018 Entrepreneurial and Innovative University Index, Turkey. Accessed (July 13, 2019): http://www.tubitak.gov.tr/sites/default/files/289/gyue_ilk50.pdf.
URAP. (2018). World Ranking Indicators. Accessed (January 15, 2018): http://www.urapcenter.org/2018/indicator.php?q=1.
Uslu, B. (2017a). Appearing in international rankings: How do academics from high-ranked universities comment? The 2nd International Higher Education Studies Conference (IHEC 2017), October 12–14. Turkey: Antalya.
Uslu, B. (2017b). The influence of organisational features in high-ranked universities: the case of Australia. Journal of Higher Education Policy and Management, 39(5), 471–486. https://doi.org/10.1080/1360080X.2017.1354755.
Uslu, B. (2018a). Dünya Üniversiteler Sıralaması: Genişletilen gösterge setine göre sıralamada oluşan farklılıklar [The World University Rankings: Differentiations in rankings according to the expanded indicator set]. Yükseköğretim ve Bilim Dergisi, 8(3), 457–470. https://doi.org/10.5961/jhes.2018.287.
Uslu, B. (2018b). The components of communication systems in universities: Their influence on academic work life. Tertiary Education and Management, 24(1), 34–48. https://doi.org/10.1080/13583883.2017.1359662.
Wang, J., & Shapira, P. (2011). Funding acknowledgement analysis: An enhanced tool to investigate research sponsorship impacts: The case of nanotechnology. Scientometrics, 87(3), 563–586. https://doi.org/10.1007/s11192-011-0362-5.
YÖK. (2017). Araştırma üniversitesi olarak belirlenen üniversiteler Cumhurbaşkanı Erdoğan tarafından açıklandı [The universities assigned as research university were announced by President Erdoğan]. Accessed (July 22, 2019): https://www.yok.gov.tr/Sayfalar/Haberler/arastirma-universitesi-olarak-belirlenen-universiteler-cumhurbaskani-tarafindan-aciklandi.aspx.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Uslu, B. A path for ranking success: what does the expanded indicator-set of international university rankings suggest?. High Educ 80, 949–972 (2020). https://doi.org/10.1007/s10734-020-00527-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10734-020-00527-0