Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India
- 705 Downloads
It is now generally accepted that institutions of higher education and research, largely publicly funded, need to be subjected to some benchmarking process or performance evaluation. Currently there are several international ranking exercises that rank institutions at the global level, using a variety of performance criteria such as research publication data, citations, awards and reputation surveys etc. In these ranking exercises, the data are combined in specified ways to create an index which is then used to rank the institutions. These lists are generally limited to the top 500–1000 institutions in the world. Further, some criteria (e.g., the Nobel Prize), used in some of the ranking exercises, are not relevant for the large number of institutions that are in the medium range. In this paper we propose a multidimensional ‘Quality–Quantity’ Composite Index for a group of institutions using bibliometric data, that can be used for ranking and for decision making or policy purposes at the national or regional level. The index is applied here to rank Central Universities in India. The ranks obtained compare well with those obtained with the h-index and partially with the size-dependent Leiden ranking and University Ranking by Academic Performance. A generalized model for the index using other variables and variable weights is proposed.
KeywordsNational ranking Research competitiveness Research ranking Scientometrics University Ranking India
This work is supported by research grants from Department of Science and Technology, Government of India (Grant: INT/MEXICO/P-13/2012) and University Grants Commission of India [Grant: F. No. 41-624/2012(SR)].
Compliance with ethical standards
Conflict of interest
- Academic Ranking of World Universities. (2011). Ranking methodology. Available at: www.arwu.org/ARWUMethodology2007.jsp. Accessed May 11, 2015.
- Basu, A., Aggarwal, R., Kumar, D., Kathuria, G., Jha, S., & Rani, T. (2000). Scientific productivity…where do we stand? Second brief report on bibliometric indicators of Indianscience, NISTADS Report, January 2000.Google Scholar
- Basu, A., & Nagpaul, P. S. (1998). National mapping of science. NISTADS Report # REP-248/98.Google Scholar
- Bornmann, L., Bowman, B. F., Bauer, J., Marx, W., Schier, H., & Palzenberger, M. (2014), Bibliometric standards for evaluating research institutes in the natural sciences. In B. Cronin, C. R. Sugimoto (Eds.), Beyond bibliometrics (pp. 201–224). Massachusetts Institute of Technology. ISBN: 978-0-262-525551-0.Google Scholar
- Bornmann, L., Moritz, S., Felix de Moya, A., & Rüdiger, M. (2014b). What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused institutions worldwide. Journal of Informetrics, 8(3), 581–593.CrossRefGoogle Scholar
- Bradford, S. C. (1934). Sources of information on specific subjects. Engineering, 137, 85–86. Reprinted in 1985 in Journal of Information Science, 10(4), 173–180.Google Scholar
- García, J. A., Rodriguez-Sánchez, R., Fdez-Valdivia, J., Torres-Salinas, D., & Herrera, F. (2012). Ranking of research output of universities on the basis of the multidimensional prestige of influential fields: Spanish universities as a case of study. Scientometrics, 93(3), 1081–1099.CrossRefGoogle Scholar
- Huang, M. H., & Chi, P. S. (2010). A comparative analysis of the application of h-index, g-index, and a-index in institutional-level research. Journal of Library and Information Studies, 8(2), 1–10.Google Scholar
- Huang, M.-H., & Lin, C. S. (2011). Counting methods and university ranking by h-index, ASIST 2011, October 9–13, 2011, New Orleans, LA, USA.Google Scholar
- Leydesdorff, L., & Opthof, T. (2010). Normalization, CWTS indicators, and the Leiden rankings: Differences in citation behavior at the level of fields. arXiv preprint arXiv:1003.3977.
- National Institutional Ranking Framework, NIRF. (2015). A methodology for ranking of universities and colleges in India. Ministry of Human Resource Development, MHRD https://www.nirfindia.org/Docs/Ranking%20Framework%20for%20Universities%20and%20Colleges.pdf. Accessed January 2016.
- UGC Note on Vacant Positions. (2014). http://www.ugc.ac.in/pdfnews/0342004_vacant-position-CU-as-on-01-01-2014.pdf. Accessed on June 18, 2014.
- Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C. M., Tijssen, R. J. W., Van Eck, N. J., et al. (2012). The Leiden ranking 2011/2012: Data collection, indicators and interpretation. Journal of the American Society of Information Science and Technology, 63(12), 2419–2432.CrossRefGoogle Scholar