Abstract
University rankings by fields are usually based on the research output of universities. However, research managers and rankings consumers expect to see in such fields a reflection of the structure of their own organizational institution. In this study we address such misinterpretation by developing the research profile of the organizational units of two Spanish universities: University of Granada and Pompeu Fabra University. We use two classification systems, the subject categories offered by Thomson Scientific which are commonly used on bibliometric studies, and the 37 disciplines displayed by the Spanish I-UGR Rankings which are constructed from an aggregation of the former. We also describe in detail problems encountered when working with address data from a top down approach and we show differences between universities structures derived from the interdisciplinary organizational forms of new managerialism at universities. We conclude by highlighting that rankings by fields should clearly state the methodology for the construction of such fields. We indicate that the construction of research profiles may be a good solution for universities for finding out levels of discrepancy between organizational units and subject fields.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323, 892–895.
Bornmann, L., Moya-Anegón, F., & Mutz, R. (2013a). Do universities or research institutions with a specific profile have an advantage or a disadvantage in institutional rankings? A latent class analysis with data from the SCImago Ranking. Journal of the American Society for Information Science and Technology. doi:10.1002/asi.22923.
Bornmann, L., Mutz, R., & Daniel, H.-D. (2013b). Multilevel-statistical reformulation of citation-based university rankings: The Leiden ranking 2011/2012. Journal of the American Society for Information Science and Technology, 64(8), 1649–1658.
Bornmann, L., Stefaner, M., Moya-Anegón, F., & Mutz, R. (2013c). Ranking and mapping of universities and research-focused institutions worldwide based on highly-cited papers: A visualization of results from multi-level models. Online Information Review. arXiv:1212.0304.
Calero-Medina, C., & van Leeuwen, T. N. (2012). Reorganizing research with the help of bibliometric collaboration networks. Case study in a University Hospital. In C. Calero-Medina (Ed.), Links in science: Linking network and bibliometric analyses in the study of research performance [Doctoral thesis] (pp. 45–66). Leiden: Universiteit Leiden.
Cheng, Y., & Liu, N. C. (2006). A first approach to the classification of the top 500 world universities by their disciplinary characteristics using scientometrics. Scientometrics, 68(1), 135–150.
Cuxac, P., Lamirel, J.-C., & Bonvallot, V. (2013). Efficient supervised and semi-supervised approaches for affiliations disambiguation. Scientometrics, 97(1), 47–58.
De Bruin, R. E., & Moed, H. F. (1993). Delimitation of scientific subfields using cognitive words from corporate addresses in scientific publications. Scientometrics, 26(1), 65–80.
Deaton, A. (1997). Analysis of household surveys. Baltimore: Johns Hopkins University Press.
García, J. A., Rodriguez-Sánchez, R., Fdez-Valdivia, J., Robinson-Garcia, N., & Torres-Salinas, D. (2012). Mapping academic institutions according to their journal publication profile: Spanish universities as a case study. Journal of the American Society for Information Science and Technology, 63(11), 2328–2340.
Gibbons, M., Limoges, C., Nowotny, H., Schwartzmann, S., Scott, P., & Trow, M. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.
Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367.
Leydesdorff, L., & Opthof, T. (2010). Scopus’s source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61(11), 2365–2369.
López-Illescas, C., Moya-Anegón, F., & Moed, H. F. (2011). A ranking of universities should account for differences in their disciplinary specialization. Scientometrics, 88(2), 563–574.
Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 381–422.
Morris, N. (2002). The developing role of departments. Research Policy, 31(5), 817–833.
Praal, F., Koosten, J., Calero-Medina, C., & Visser, M. S. (2013). Ranking universities: The challenge of affiliated institutes. In 18th International Conference on Science and Technology Indicators.
Pudovkin, A. I., & Garfield, E. (2002). Algorithmic procedure for finding semantically related journals. Journal of the American Society for Information Science and Technology, 53(13), 1113–1119.
Rescher, N. (2006). Epistemetrics. Cambridge: Cambridge University Press.
Robinson-García, N., Moreno-Torres, J. G., Torres-Salinas, D., López-Cózar, E. D., & Herrera, F. (2013a). The role of national university rankings in an international context: The case of the I-UGR Rankings of Spanish Universities. In Conference Proceedings from the 14th International Society for Scientometrics and Informetrics Conference (Vol. 2, pp. 1550–1565).
Robinson-García, N., Rodriguez-Sanchez, R., García, J. A., Torres-Salinas, D., & Fdez-Valdivia, J. (2013b). Análisis de redes de las universidades españolas de acuerdo a su perfil de publicación en revistas por áreas científicas. Revista Española de Documentación Científica (in press).
Torres-Salinas, D., Moreno-Torres, J. G., López-Cózar, E. D., & Herrera, F. (2011). A methodology for institution-field ranking based on a bidimensional analysis: The IFQ2A-index. Scientometrics, 88(3), 771–786.
Van Leeuwen, T. N. (2007). Modeling of bibliometric approaches and importance of output verification in research performance assessment. Research Evaluation, 16(2), 93–105.
Van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.
Waltman, L., & van Eck, N. J. (2012). Source normalized indicators of citation impact: An overview of different approaches and an empirical comparison. Scientometrics, 96(3), 699–716.
Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E., Tijssen, R. J. W., van Eck, N. J., et al. (2012). The Leiden Ranking 2011/2012: Data collection indicators and interpretation. Journal of the American Society for Information Science and Technology, 63(12), 2419–2432.
Acknowledgments
Thanks are due to the two anonymous referees for their constructive suggestions. The authors would also like to thank Thed N. van Leeuwen and Daniel Torres-Salinas for their helpful comments on previous versions of this paper. Nicolás Robinson-García is currently supported by a FPU Grant from the Spanish Ministerio de Economía y Competitividad.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Robinson-García, N., Calero-Medina, C. What do university rankings by fields rank? Exploring discrepancies between the organizational structure of universities and bibliometric classifications. Scientometrics 98, 1955–1970 (2014). https://doi.org/10.1007/s11192-013-1157-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-013-1157-7