What do university rankings by fields rank? Exploring discrepancies between the organizational structure of universities and bibliometric classifications
University rankings by fields are usually based on the research output of universities. However, research managers and rankings consumers expect to see in such fields a reflection of the structure of their own organizational institution. In this study we address such misinterpretation by developing the research profile of the organizational units of two Spanish universities: University of Granada and Pompeu Fabra University. We use two classification systems, the subject categories offered by Thomson Scientific which are commonly used on bibliometric studies, and the 37 disciplines displayed by the Spanish I-UGR Rankings which are constructed from an aggregation of the former. We also describe in detail problems encountered when working with address data from a top down approach and we show differences between universities structures derived from the interdisciplinary organizational forms of new managerialism at universities. We conclude by highlighting that rankings by fields should clearly state the methodology for the construction of such fields. We indicate that the construction of research profiles may be a good solution for universities for finding out levels of discrepancy between organizational units and subject fields.
KeywordsUniversity rankings Fields Address data Institutional structure Subject classification
Thanks are due to the two anonymous referees for their constructive suggestions. The authors would also like to thank Thed N. van Leeuwen and Daniel Torres-Salinas for their helpful comments on previous versions of this paper. Nicolás Robinson-García is currently supported by a FPU Grant from the Spanish Ministerio de Economía y Competitividad.
- Bornmann, L., Moya-Anegón, F., & Mutz, R. (2013a). Do universities or research institutions with a specific profile have an advantage or a disadvantage in institutional rankings? A latent class analysis with data from the SCImago Ranking. Journal of the American Society for Information Science and Technology. doi: 10.1002/asi.22923.
- Bornmann, L., Stefaner, M., Moya-Anegón, F., & Mutz, R. (2013c). Ranking and mapping of universities and research-focused institutions worldwide based on highly-cited papers: A visualization of results from multi-level models. Online Information Review. arXiv:1212.0304.Google Scholar
- Calero-Medina, C., & van Leeuwen, T. N. (2012). Reorganizing research with the help of bibliometric collaboration networks. Case study in a University Hospital. In C. Calero-Medina (Ed.), Links in science: Linking network and bibliometric analyses in the study of research performance [Doctoral thesis] (pp. 45–66). Leiden: Universiteit Leiden.Google Scholar
- Cuxac, P., Lamirel, J.-C., & Bonvallot, V. (2013). Efficient supervised and semi-supervised approaches for affiliations disambiguation. Scientometrics, 97(1), 47–58.Google Scholar
- García, J. A., Rodriguez-Sánchez, R., Fdez-Valdivia, J., Robinson-Garcia, N., & Torres-Salinas, D. (2012). Mapping academic institutions according to their journal publication profile: Spanish universities as a case study. Journal of the American Society for Information Science and Technology, 63(11), 2328–2340.CrossRefGoogle Scholar
- Gibbons, M., Limoges, C., Nowotny, H., Schwartzmann, S., Scott, P., & Trow, M. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.Google Scholar
- Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 381–422.Google Scholar
- Praal, F., Koosten, J., Calero-Medina, C., & Visser, M. S. (2013). Ranking universities: The challenge of affiliated institutes. In 18th International Conference on Science and Technology Indicators.Google Scholar
- Robinson-García, N., Moreno-Torres, J. G., Torres-Salinas, D., López-Cózar, E. D., & Herrera, F. (2013a). The role of national university rankings in an international context: The case of the I-UGR Rankings of Spanish Universities. In Conference Proceedings from the 14th International Society for Scientometrics and Informetrics Conference (Vol. 2, pp. 1550–1565).Google Scholar
- Robinson-García, N., Rodriguez-Sanchez, R., García, J. A., Torres-Salinas, D., & Fdez-Valdivia, J. (2013b). Análisis de redes de las universidades españolas de acuerdo a su perfil de publicación en revistas por áreas científicas. Revista Española de Documentación Científica (in press).Google Scholar
- Waltman, L., & van Eck, N. J. (2012). Source normalized indicators of citation impact: An overview of different approaches and an empirical comparison. Scientometrics, 96(3), 699–716.Google Scholar