World’s Best Universities and Personalized Rankings

  • Mario Inostroza-Ponta
  • Natalie Jane de Vries
  • Pablo Moscato
Living reference work entry

Later version available View entry history


This chapter presents a heuristic for a multi-objective ranking problem using a dataset of international interest as an example of its application, namely, the ranking of the world’s top educational institutions. The problem of ranking academic institutions is a subject of keen interest for administrators, consumers, and research policy makers. From a mathematical perspective, the proposed heuristic addresses the need for more transparent models and associated methods related to the problem of identifying sound relative rankings of objects with multiple attributes. The low complexity of the method allows software implementations that scale well for thousands of objects as well as permitting reasonable visualization. It is shown that a simple and multi-objective-aware ranking system can easily be implemented, which naturally leads to intuitive research policies resulting from varying scenarios presented within. The only assumption that this method relies on is the ability to sort the candidate objects according to each given attribute. Thus the attributes could be numerical or ordinal in nature. This helps to avoid the selection of an ad hoc single score based on an arbitrary assignment of attributes’ weights as other heuristics do. To illustrate the use of this proposed methodology, results are presented and obtained using the dataset on the ranking of world universities (of the years 2007–2012), by academic performance, published annually by ARWU.


Analytics Digital humanities Pareto optimality Ranking Symbolic regression 



Pablo Moscato acknowledges support from the Australian Research Council Future Fellowship FT120100060 and Australian Research Council Discovery Projects DP120102576 and DP140104183.


  1. 1.
    Adler NJ, Harzing AW (2009) When knowledge wins: transcending the sense and nonsense of academic rankings. Acad Manag Learn Edu 8(1):72–95. doi10.5465/AMLE.2009.37012181,,
  2. 2.
    Agnew T, Whitlock R, Neutze J, Kerr A (1994) Waiting lists for coronary artery surgery: can they be better organised? N Z Med J 107(979):211–215Google Scholar
  3. 3.
    ARWU (2013) ARWU methodology: definition of indicators.
  4. 4.
    Bang-Jensen J, Gutin G (2001) Digraphs: theory, algorithms and applications. Springer, LondonzbMATHGoogle Scholar
  5. 5.
    Baty P (2014) Caltech: secrets of the worlds number one university: how does a tiny institution create such outsized impact?
  6. 6.
    Braga-Neto U, Hashimoto R, Dougherty ER, Nguyen DV, Carroll RJ (2004) Is cross-validation better than resubstitution for ranking genes? Bioinformatics 20(2):253–258. doi10.1093/bioinformatics/btg399,
  7. 7.
    Buela-Casal G, Gutirrez-Martnez O, Bermdez-Snchez M, Vadillo-Muoz O (2007) Comparative study of international academic rankings of universities. Scientometrics 71(3):349–365. doi10.1007/s11192-007-1653-8,
  8. 8.
    Burden DJ (1995) The ranking of dental aesthetics. J Orthod 22(3):259–261. CrossRefGoogle Scholar
  9. 9.
    Carrillo VM, Taboada HA (2012) A general iterative procedure of the non-numerical ranking preferences method for multiple objective decision making. In: Proceedings of the complex adaptive systems 2012 conference, Washington, DC, pp 135–139. doi10.1016/j.procs.2012.09.043,
  10. 10.
    Chen Y, Kilgour DM, Hipel KW (2011) An extreme-distance approach to multiple criteria ranking. Math Comput Model 53(5–6):646–658. doi10.1016/j.mcm.2010.10.001,
  11. 11.
    Dadelo S, Turskis Z, Zavadskas EK, Dadeliene R (2014) Multi-criteria assessment and ranking system of sport team formation based on objective-measured values of criteria set. Expert Syst Appl 41(14):6106–6113. doi10.1016/j.eswa.2014.03.036,
  12. 12.
    Dalal O, Sengamedu SH, Sanyal S (2012) Multi-objective ranking of comments on web. In: Proceedings of the 21st World Wide Web conference, WWW 2012, Lyon, pp 419–428. doi10.1145/2187836.2187894,
  13. 13.
    DeOliveira E, Callum R (2004) Who’s the best? Data envelopment analysis and ranking players in the national football league. In: Economics, management and optimization in sports, 1st edn. Springer, Berlin/Heidelberg, pp 15–30. doi10.1007/978-3-540-24734-0Google Scholar
  14. 14.
    Docampo D (2013) Reproducibility of the Shanghai academic ranking of world universities results. Scientometrics 94(2):567–587. doi10.1007/s11192-012-0801-y,
  15. 15.
    Fernandez E, Leyva JC (2002) A method based on multiobjective optimization for deriving a ranking from a fuzzy preference relation. Eur J Oper Res 154(1):110–124. doi10.1016/S0377-2217(02)00705-1,
  16. 16.
    Gehlbach FR (1975) Investigation, evaluation, and priority ranking of natural areas. Biol Conserv 8(2):79–88. doi10.1016/0006-3207(75)90033-6,
  17. 17.
    Geneletti D, van Duren I (2008) Protected area zoning for conservation and use: a combination of spatial multicriteria and multiobjective evaluation. Landsc Urban Plan 85(2):97–110. doi10.1016/j.landurbplan.2007.10.004,
  18. 18.
    Gerani S, Zhai C, Crestani F (2012) Score transformation in linear combination for multi-criteria relevance ranking. In: Advances in information retrieval – proceedings of 34th European conference on IR research, ECIR 2012, Barcelona, pp 256–267Google Scholar
  19. 19.
    Greco S, Mousseau V, Slowinski R (2008) Ordinal regression revisited: multiple criteria ranking using a set of additive value functions. Eur J Oper Res 191(2):416–436. doi10.1016/j.ejor.2007.08.013,
  20. 20.
    Harvey L, Green D (1993) Defining quality. Assess Eval High Educ 18(1):9–34. doi10.1080/0260293930180102,
  21. 21.
    Hinloopen E, Nijkamp P, Rietveld P (2004) Integration of ordinal and cardinal information in multi-criteria ranking with imperfect compensation. Eur J Oper Res 158(2):317–338. doi10.1016/j.ejor.2003.06.007,
  22. 22.
    Hu ZH, Zhou JX, Zhang MJ, Zhao Y (2015) Methods for ranking college sports coaches based on data envelopment analysis and pagerank. Expert Syst 32(6):652–673. doi10.1111/exsy.12108,, eXSY-Jun-14-124.R1
  23. 23.
    Hughes EJ (2001) Evolutionary multi-objective ranking with uncertainty and noise. In: Evolutionary multi-criterion optimization, proceedings of first international conference, EMO 2001, Zurich, pp 329–343. doi10.1007/3-540-44719-9_23,
  24. 24.
    Ioannidis J, Patsopoulos N, Kavvoura F, Tatsioni A, Evangelou E, Kouri I, Contopoulos-Ioannidis D, Liberopoulos G (2007) International ranking systems for universities and institutions: a critical appraisal. BMC Medicine 5(1):30. doi10.1186/1741-7015-5-30,
  25. 25.
    Jeremic V, Bulajic M, Martic M, Radojicic Z (2011) A fresh approach to evaluating the academic ranking of world universities. Scientometrics 87(3):587–596. doi10.1007/s11192-011-0361-6Google Scholar
  26. 26.
    Kadzinski M, Tervonen T (2013) Robust multi-criteria ranking with additive value models and holistic pair-wise preference statements. Eur J Oper Res 228(1):169–180. doi10.1016/j.ejor.2013.01.022,
  27. 27.
    Kadzinski M, Greco S, Slowinski R (2012) Selection of a representative value function in robust multiple criteria ranking and choice. Eur J Oper Res 217(3):541–553. doi10.1016/j.ejor.2011.09.032,
  28. 28.
    Karminsky A, Polozov A (2016) Evolution of ideas about rating and ranking in sports. Springer International Publishing, Cham, pp 187–200. doi10.1007/978-3-319-39261-5_7,
  29. 29.
    Kee D, Karwowski W (2003) Ranking systems for evaluation of joint and joint motion stressfulness based on perceived discomforts. Appl Ergon 34(2):167–176. doi,
  30. 30.
    Kellenberger E, Foata N, Rognan D (2008) Ranking targets in structure-based virtual screening of three-dimensional protein libraries: methods and problems. J Chem Inf Model 48(5):1014–1025. doi10.1021/ci800023x,
  31. 31.
    Lerche DB, Brüggemann R, Sørensen PB, Carlsen L, Nielsen OJ (2002) A comparison of partial order technique with three methods of multi-criteria analysis for ranking of chemical substances. J Chem Inf Comput Sci 42(5):1086–1098. doi10.1021/ci010268p,
  32. 32.
    Li W, Suh YJ, Zhang J (2006) Does logarithm transformation of microarray data affect ranking order of differentially expressed genes? In: 28th annual international conference of the IEEE Engineering in medicine and biology society, EMBS’06, vol Supplement, pp 6593–6596. doi10.1109/IEMBS.2006.260896Google Scholar
  33. 33.
    Liefner I (2003) Funding, resource allocation, and performance in higher education systems. High Educ 46(4):469–489. doi10.1023/A:1027381906977Google Scholar
  34. 34.
    Liu NC, Cheng Y (2005) The academic ranking of world universities. High Educ Eur 30(2):127–136. doi10.1080/03797720500260116,
  35. 35.
    López JCL, Aguilera-Contreras MA (2005) A multiobjective evolutionary algorithm for deriving final ranking from a fuzzy outranking relation. In: Evolutionary multi-criterion optimization, proceedings of third international conference, EMO 2005, Guanajuato, pp 235–249. doi10.1007/978-3-540-31880-4_17,
  36. 36.
    López JCL, Chavira DAG, Noriega JJS (2014) A multiobjective genetic algorithm based on NSGA II for deriving final ranking from a medium-sized fuzzy outranking relation. In: 2014 IEEE symposium on computational intelligence in multi-criteria decision-making, MCDM 2014, Orlando, pp 24–31. doi10.1109/MCDM.2014.7007184,
  37. 37.
    Machado EA, Stoms DM, Davis FW, Kreitler J (2006) Prioritizing farmland preservation cost-effectively for multiple objectives. J Soil Water Conserv 61(5):250–258. Google Scholar
  38. 38.
    Malekmohammadi B, Zahraie B, Kerachian R (2011) Ranking solutions of multi-objective reservoir operation optimization models using multi-criteria decision analysis. Expert Syst Appl 38(6):7851–7863. doi10.1016/j.eswa.2010.12.119,
  39. 39.
    Marginson S (2007) Global university rankings: implications in general and for Australia. J High Educ Policy Manag 29:131–142. doi10.1080/13600800701351660,
  40. 40.
    Merisotis J, Sadlak J (2005) Higher education rankings: evolution, acceptance, and dialogue. High Educ Eur 30:97–101. doi10.1080/03797720500260124,
  41. 41.
    Moyles DM, Thompson GL (1969) An algorithm for finding a minimum equivalent graph of a digraph. J ACM 16(3):455–460. doi10.1145/321526.321534,
  42. 42.
    Nguyen LT, Yee WG, Liew R, Frieder O (2010) Experiences with using SVM-based learning for multi-objective ranking. In: Proceedings of the 19th ACM conference on information and knowledge management, CIKM 2010, Toronto, pp 1917–1920. doi10.1145/1871437.1871763,
  43. 43.
    Nijmegen RU (2011) Nobel prize in physics 2010.
  44. 44.
    Nwamadi O, Zhu X, Nandi AK (2012) Multi-criteria ranking based greedy algorithm for physical resource block allocation in multi-carrier wireless communication systems. Signal Process 92(11):2706–2717. doi10.1016/j.sigpro.2012.04.020,
  45. 45.
    Oddershede J (2013) What rankings dont tell you about university excellence.
  46. 46.
    di Pierro F, Khu S, Savic DA (2007) An investigation on preference order ranking scheme for multiobjective evolutionary optimization. IEEE Trans Evol Comput 11(1):17–45. doi10.1109/TEVC.2006.876362,
  47. 47.
    Pulugurtha SS, Krishnakumar VK, Nambisan SS (2007) New methods to identify and rank high pedestrian crash zones: an illustration. Accid Anal Prev 39(4):800–811. doi,
  48. 48.
    Rad A, Naderi B, Soltani M (2011) Clustering and ranking university majors using data mining and AHP algorithms: a case study in Iran. Expert Syst Appl 38(1):755–763. doi10.1016/j.eswa.2010.07.029,
  49. 49.
    Radicchi F (2011) Who is the best player ever? A complex network analysis of the history of professional tennis. PLoS ONE 6(2):1–7. doi10.1371/journal.pone.0017249,
  50. 50.
    Razvan F V (2007) Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics 72(1):25–32. doi10.1007/s11192-007-1712-1,
  51. 51.
    Reba MNM, Rosli AZ, Makhfuz MA, Sabarudin NS, Roslan NH (2013) Determination of sustainable land potential based on priority ranking: multi-criteria analysis (MCA) technique. In: Computational science and its applications – ICCSA 2013 – proceedings of 13th international conference, Ho Chi Minh City, part VI, pp 212–218. doi10.1109/ICCSA.2013.44,
  52. 52.
    Saisana M, dHombres B, Saltelli A (2011) Rickety numbers: volatility of university rankings and policy implications. Res Policy 40(1):165–177. doi10.1016/j.respol.2010.09.003,
  53. 53.
    Schall D (2014) A multi-criteria ranking framework for partner selection in scientific collaboration environments. Decis Support Syst 59:1–14. doi10.1016/j.dss.2013.10.001,
  54. 54.
    Schmidt M, Lipson H (2014) Eureqa (version 0.98 beta) [software].
  55. 55.
    Shi Z, Hao F (2013) A strategy of multi-criteria decision-making task ranking in social-networks. J Supercomput 66(1):556–571. doi10.1007/s11227-013-0934-7,
  56. 56.
    Smith PGR, Theberge JB (1987) Evaluating natural areas using multiple criteria: theory and practice. Environ Manag 11(4):447–460. doi10.1007/BF01867653,
  57. 57.
    Smits G, Kotanchek M (2005) Pareto-front exploitation in symbolic regression. In: OReilly UM, Yu T, Riolo R, Worzel B (eds) Genetic programming theory and practice II. Genetic programming, vol 8. Springer, pp 283–299. doi10.1007/0-387-23254-0_17,
  58. 58.
    Sun J, Kuo PH, Riley BP, Kendler KS, Zhao Z (2008) Candidate genes for schizophrenia: a survey of association studies and gene ranking. Am J Med Genet B Neuropsychiatr Genet 147B(7):1173–1181. doi10.1002/ajmg.b.30743,
  59. 59.
    Thakur M (2007) The impact of ranking systems on higher education and its stakeholders. J Inst Res 13(1):83–96MathSciNetGoogle Scholar
  60. 60.
    THE (2013) Times higher education: world university rankings.
  61. 61.
    Toma I, Roman D, Fensel D, Sapkota B, Gómez JM (2007) A multi-criteria service ranking approach based on non-functional properties rules evaluation. In: Proceedings of fifth international conference on service-oriented computing – ICSOC 2007, Vienna, pp 435–441. doi10.1007/978-3-540-74974-5_40,
  62. 62.
    Vladislavleva EJ dHD Smits GF (2009) Order of nonlinearity as a complexity measure for models generated by symbolic regression via Pareto genetic programming. IEEE Trans Evol Comput 13(2):333–349. doi10.1109/TEVC.2008.926486,
  63. 63.
    Voll CA, Goodwin JE, Pitney WA (1999) Athletic training education programs: to rank or not to rank? J Athl Train 34(1):48–52. Google Scholar
  64. 64.
    de Vries NJ, Carlson J, Moscato P (2014) A data-driven approach to reverse engineering customer engagement models: towards functional constructs. PLoS ONE 9(7):e102,768. doi10.1371/journal.pone.0102768,
  65. 65.
    Wu J, Liang L (2012) A multiple criteria ranking method based on game cross-evaluation approach. Annals OR 197(1):191–200. doi10.1007/s10479-010-0817-8,
  66. 66.
    Xu X (2001) The SIR method: a superiority and inferiority ranking method for multiple criteria decision making. Eur J Oper Res 131(3):587–602. doi10.1016/S0377-2217(00)00101-6,

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Mario Inostroza-Ponta
    • 1
  • Natalie Jane de Vries
    • 2
  • Pablo Moscato
    • 2
  1. 1.Departamento de Ingeniería InformáticaUniversidad de SantiagoSantiagoChile
  2. 2.School of Electrical Engineering and ComputingFaculty of Engineering and Built EnvironmentCallaghanAustralia

Personalised recommendations