Scientometrics

, Volume 88, Issue 3, pp 771–786 | Cite as

A methodology for Institution-Field ranking based on a bidimensional analysis: the IFQ2A index

  • Daniel Torres-Salinas
  • Jose G. Moreno-Torres
  • Emilio Delgado-López-Cózar
  • Francisco Herrera
Article

Abstract

The problem of comparing academic institutions in terms of their research production is nowadays a priority issue. This paper proposes a relative bidimensional index that takes into account both the net production and the quality of it, as an attempt to provide a comprehensive and objective way to compare the research output of different institutions in a specific field, using journal contributions and citations. The proposed index is then applied, as a case study, to rank the top Spanish universities in the fields of Chemistry and Computer Science in the period ranging from 2000 until 2009. A comparison with the top 50 universities in the ARWU rankings is also made, showing the proposed ranking is better suited to distinguish among non-elite universities.

Keywords

Rankings Universities Higher education Bibliometrics Shanghai ranking Bidimensional analysis Evaluation models Research performance assessment h-index 

References

  1. Adler, N., & Harding, A.-W. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. The Academy of Management Learning and Education, 8(1), 72–95.CrossRefGoogle Scholar
  2. Aguillo, I. F., Bar-Ilan, J., Levene, M., & Priego, J. L. O. (2010). Comparing university rankings. Scientometrics, 85(1), 243–256.CrossRefGoogle Scholar
  3. Aksnes, D. (2003). Characteristics of highly cited papers. Research Evaluation, 12, 159–170.CrossRefGoogle Scholar
  4. Aksnes, D., & Sivertsen, G. (2004). The effect of highly cited papers on national citation indicators. Scientometrics, 59, 213–224.CrossRefGoogle Scholar
  5. Alonso, S., Cabrerizo, F., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273–289.CrossRefGoogle Scholar
  6. Billaut, J.-C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shanghai ranking?: An MCDM view. Scientometrics, 84(1), 237–263.CrossRefGoogle Scholar
  7. Bornmann, L., & Daniel, H.-D. (2008). Selecting manuscripts for a high-impact journal through peer review: A citation analysis of communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology, 59(11), 1841–1852.CrossRefGoogle Scholar
  8. Bornmann, L., & Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.CrossRefGoogle Scholar
  9. Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). The h index research output measurement: Two approaches to enhance its accuracy. Journal of Informetrics, 4(3), 407–414.Google Scholar
  10. Buela-Casal, G., Gutiérrez-Martínez, O., Bermúdez-Sánchez, M. P., & Vadillo-Muñoz, O. (2007). Comparative study of international academic rankings of universities. Scientometrics, 71(3), 349–365.CrossRefGoogle Scholar
  11. Cabrerizo, F., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.CrossRefGoogle Scholar
  12. Dehon, C., McCathie, A., & Verardi, V. (2010). Uncovering excellence in academic rankings: A closer look at the Shanghai ranking. Scientometrics, 83(2), 515–524.CrossRefGoogle Scholar
  13. Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems. Higher Education 49, 495–533. doi:10.1007/s10734-004-1746-8.
  14. Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.MathSciNetCrossRefGoogle Scholar
  15. Elsevier B.V. Scopus. (2010). See http://www.scopus.co. Accessed Oct 2010.
  16. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences USA, 102(46), 16569–16572.CrossRefGoogle Scholar
  17. ISI Web of Science. Science Citation Index. See http://isiknowledge.co. Accessed Oct 2010.
  18. Kosmulski, M. (2009). New seniority-independent Hirsch-type index. Journal of Informetrics, 3(4), 341–347.CrossRefGoogle Scholar
  19. Leydesdorff, L., & Opthof, T. (2010). Normalization, CWTS indicators, and the Leiden Rankings: Differences in citation behavior at the level of fields. http://www.leydesdorff.net/reply2cwts/reply2cwts.pd. Accessed Oct 2010.
  20. Liu, N. C., & Cheng, Y. (2005). Academic ranking of world universities: Methodologies and problems. Higher Education in Europe 30(2), 127–136.Google Scholar
  21. Moed, H. F. (2005). Citation analysis in research evaluation. New York: Springer.Google Scholar
  22. Moed, H. F. (2010). CWTS crown indicator measures citation impact of a research group’s publication oeuvre. CoRR, abs/1003.5884.Google Scholar
  23. Price, D. J. D. S. (1965). Networks of scientific papers. Science, 149(3683), 510–515.CrossRefGoogle Scholar
  24. QS World University Rankings. (2010). http://www.topuniversities.com/university-rankings/world-university-rankings/hom. Accessed Oct 2010.
  25. SCImago Institutions Rankings. (2007). http://www.scimagoir.co. Accessed Oct 2010.
  26. Shanghai Jiao Tong University. ( 2009). Academic ranking of world universities (ARWU). http://www.arwu.org/index.js. Accessed Oct 2010.
  27. Sypsa, V., & Hatzakis, A. (2009). Assessing the impact of biomedical research in academic institutions of disparate sizes. BMC Medical Research Methodology, 9, 33–42.CrossRefGoogle Scholar
  28. Universiteit Leiden. (2010). Leiden university rankings. http://www.socialsciences.leiden.edu/cwts/products-services/leiden-ranking-2010-cwts. Accessed Oct 2010.
  29. van Raan, A. F. J., van Leeuwen, T. N., Visser, M. S., van Eck, N. J., & Waltman, L. (2010). Rivals for the crown: Reply to Opthof and Leydesdorff. Journal of Informetricx, 4(3), 431–435.CrossRefGoogle Scholar
  30. Vieira, E., & Gomes, J. (2010). A research impact indicator for institutions. Journal of Informetrics, 4(4), 581–590.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2011

Authors and Affiliations

  • Daniel Torres-Salinas
    • 1
  • Jose G. Moreno-Torres
    • 2
  • Emilio Delgado-López-Cózar
    • 3
  • Francisco Herrera
    • 2
  1. 1.EC3: Evaluación de la Ciencia y la Comunicación Científica, Centro de Investigación Médica AplicadaUniversidad de NavarraNavarraSpain
  2. 2.Department of Computer Science and Artificial IntelligenceUniversidad de GranadaGranadaSpain
  3. 3.EC3: Evaluación de la Ciencia y la Comunicación Científica, Departamento de Biblioteconomía y DocumentaciónUniversidad de GranadaGranadaSpain

Personalised recommendations