Benchmarking research performance at the university level with information theoretic measures
- 446 Downloads
This paper presents a new method for comparing universities based on information theoretic measures. The research output of each academic institution is represented statistically by an impact-factor histogram. To this aim, for each academic institution we compute the probability of occurrence of a publication with impact factor in different intervals. Assuming the probabilities associated with a pair of academic institutions our objective is to measure the Information Gain between them. To do so, we develop an axiomatic characterization of relative information for predicting institution-institution dissimilarity. We use the Spanish university system as our scenario to test the proposed methodology for benchmarking three universities with the rest as a case study. For each case we use different scientific fields such as Information and Communication Technologies, Medicine and Pharmacy, and Economics and Business as we believe comparisons must take into account their disciplinary context. Finally we validate the Information Gain values obtained for each case with previous studies.
KeywordsInformation gain Institution-institution similarity Impact-factor histogram Information theoretic measure Information conservation constraint Benchmarking research output
This research was sponsored by the Spanish Board for Science and Technology (MICINN) under grant TIN2010-15157 cofinanced with European FEDER funds. Nicolás Robinson-García is currently supported by a FPU grant from the Spanish Ministerio de Educación y Ciencia. Thanks are due to the reviewers for their constructive suggestions.
- Fisher, R. A., (1950). The logic of inductive inference. Journal of the Royal Statistical Society, 98, 39–54; Contributions to Mathematical Statistics. John Wiley and Sons, New York, paper 26.Google Scholar
- García, J. A., Rodríguez-Sánchez, R., Fdez-Valdivia, J., Robinson-García, N. & Torres-Salinas. (2012a). Mapping academic institutions according to their journal publication profile: Spanish universities as a case study. Journal of the American Society for Information Science and Technology. Accepted for publication.Google Scholar
- García, J. A., Rodríguez-Sánchez, R., Fdez-Valdivia, J., Torres-Salinas, D. & Herrera, F. (2012b). Ranking of research output of universities on the basis of the multidimensional prestige of influential fields: Spanish universities as a case of study. Scientometrics. doi: 10.1007/s11192-012-0740-7.
- Kullback, S. (1978). Information theory and statistics. Gloucester, Mass: Peter Smith.Google Scholar
- Lundberg, J. (2006). Bibliometrics as a research assessment tool impact beyond the impact factor [PhD dissertation]. Stockholm: Karolinska Institutet.Google Scholar
- Ortega, J. L., Lopez-Romero, E. & Fernandez, I. (2011). Multivariate approach to classify research institutes according to their outputs: The case of the CSIC’s institutes. Journal of Informetrics, 5(3), 323–332.Google Scholar
- Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423; 623–656.Google Scholar
- Tijssen, R. J. W., van Leeuwen, T. N. & van Wijk, E. (2009). Benchmarking university-industry research cooperation worldwide: performance measurements and indicators based on co-authorship data for the world’s largest universities. Research Evaluation, 18(1), 13–24.Google Scholar
- Wiener, N. (1950). The Human use of human beings. Boston: Houghton Mifflin Co.Google Scholar