Skip to main content
SpringerLink
Log in

Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014)

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

We present a novel way to frame a discussion of changes in publication patterns that occur in a context of performance-based research funding systems (PRFSs). Adopting an approach derived from social epistemology, we foreground the dialectical nature of knowledge. This allows us to relate changes in publication patterns to PRFS and show the tensions that emerge between relatively diverse research fields with diverse publication practices and bibliometric indicators within PRFSs that reinforce a singular view of research goals. Specifically, we highlight that the employment of bibliometric indicators result in a fixed hierarchy among communication media that may be at odds with the goals within research fields subjected to PRFSs. These ideas are illustrated with an empirical analysis of changes in publication patterns within the field of educational research at the University of Gothenburg (GU; 2005–2014) in Sweden. We contrast the observed changes with implicit priorities in the national and institutional PRFS that operate in this context since 2009. Findings from bibliometric analysis indicate a move away from publication traditions that used to be characteristic of educational research: the growth in the number of journal articles is greater than that in the number of book chapters, while the number of reports is on a declining slope. In relation to PRFSs, we show that conclusive judgement on the desirability of the observed changes is hardly achievable. If one adopts the aims implicit in PRFS, research performance appears to be enhanced. If one sides with the views of many educational researchers, then some of the trends might be an indication of undesirable changes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Aagaard, K. (2015). How incentives trickle down: Local use of a national bibliometric indicator system. Science and Public Policy, 42(5), 725–737. https://doi.org/10.1093/scipol/scu087.

    Article  Google Scholar 

  • Aagaard, K., Bloch, C., & Schneider, J. W. (2015). Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator. Research Evaluation, 24(2), 106–117.

    Article  Google Scholar 

  • Aldberg, H., & Jacobsson, C. (2014). Bibliometriskt underlag för medelsfördelning. Vetenskapsrådet. https://www.vr.se/download/18.5a947f0d145b21c1709f9d/1399897207262/Bibliometriskt+underlag+f%C3%B6r+medelsf%C3%B6rdelning_140512.pdf. Accessed 20 Mar 2016.

  • Angervall, P., & Gustafsson, J. (2014). Becoming an Academic Researcher. Policy Futures in Education, 12(2), 191–199.

    Article  Google Scholar 

  • Anslagsfördelning och kostnadsdebitering för budgetåret 2015 samt planeringsramar för 2016–2017., Pub. L. No. E 2014/257 § Universitetsstyrelsen (2014). http://medarbetarportalen.gu.se/digitalAssets/1497/1497483_anslagsf–rdelning-och-kostnadsdebitering-2015-inkl-bilagor.pdf. Accessed 22 Apr 2016.

  • Butler, L. (2003). Modifying publication practices in response to funding formulas. Research Evaluation, 12(1), 39–46.

    Article  Google Scholar 

  • Butler, L. (2004). What happens when funding is linked to publication counts? In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research: the use of publication and patent statistics in studies of S&T systems (pp. 389–405). Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  • Carlsson, H. (2009). Allocation of Research Funds Using Bibliometric Indicators—Asset and Challenge to Swedish Higher Education Sector. InfoTrend, 64(4), 82–88.

    Google Scholar 

  • de Rijcke, S., Wouters, P. F., Rushforth, A. D., Franssen, T. P., & Hammarfelt, B. (2015). Evaluation practices and effects of indicator use—a literature review. Research, Evaluation, rvv038.

    Google Scholar 

  • Dees, W. (2008). Innovative scientometric methods for a continuous monitoring of research activities in educational science. In H. Kretschmer & F. Havemann (Eds.), Proceedings of WIS 2008, Berlin (pp. 1–10). Berlin.

  • Dougherty, K. J., Jones, S. M., Lahr, H., Pheatt, L., Natow, R. S., & Reddy, V. (2016). Performance funding for higher education. Baltimore: Johns Hopkins University Press.

    Google Scholar 

  • Engels, T. C. E., Ossenblok, T. L. B., & Spruyt, E. H. J. (2012). Changing publication patterns in the Social Sciences and Humanities, 2000–2009. Scientometrics, 93(2), 373–390.

    Article  Google Scholar 

  • Englund, T. (2006). New Trends in Swedish Educational Research. Scandinavian Journal of Educational Research, 50(4), 383–396.

    Article  Google Scholar 

  • Ett lyft för forskning och innovation, Pub. L. No. Regeringens proposition 2008/09:50 (2008). http://www.regeringen.se/contentassets/05cb6c62a34e4b37a114611a3ebcbd5b/ett-lyft-for-forskning-och-innovation-prop.-20080950. Accessed 10 Jan 2014.

  • Fuller, S. (2002). Social epistemology (2nd ed.). Bloomington: Indiana University Press.

    Google Scholar 

  • Gläser, J. (2017). A fight on epistemological quicksand: Comment on the dispute between van den Besselaar et al. and Butler. Journal of Informetrics, 11(3), 927–932.

    Article  Google Scholar 

  • Gothenburg University Library: Allocation. (2015). http://www.ub.gu.se/publicera/bibliometri/resursfordelning/ Accessed 4 Dec 2015.

  • Gothenburg University Library: Publication types in GUP. (2015). http://www.ub.gu.se/publicera/publikationsbas/publtyper/. Accessed 26 Jan 2016.

  • Hammarfelt, B., & de Rijcke, S. (2015). Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation, 24(1), 63–77.

    Article  Google Scholar 

  • Hansen, M., & Lindblad, S. (2010). Forskningskommunikation och Publiceringsmönsrwe inom Utbildningsvetenskap (VETENSKAPSRÅDETS RAPPORTSERIE) (p. 95). Stockholm: Vetenskapsrådet. https://publikationer.vr.se/produkt/forskningskommunikation-och-publiceringsmonster-inom-utbild/ Accessed 14 Oct 2015.

  • Hicks, D. (2004). The Four Literatures of Social Science. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 473–496). Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  • Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251–261.

    Article  Google Scholar 

  • Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520, 429–431.

    Article  Google Scholar 

  • Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of “new” performance funding in higher education. Educational Evaluation and Policy Analysis, 37(4), 501–519.

    Article  Google Scholar 

  • Inriktningsbeslut för budgetarbetet 2012. Sammanfattning., Pub. L. No. V 2011/190 (2011). http://medarbetarportalen.gu.se/digitalAssets/1535/1535388_inriktningsbeslut-f–r-budgetarbetet-2012.pdf. Accessed 4 Apr 2016.

  • Jiménez-Contreras, E., de Moya Anegón, F., & López-Cózar, E. D. (2003). The evolution of research activity in Spain. Research Policy, 32(1), 123–142.

    Article  Google Scholar 

  • Jonkers, K., & Zacharewicz, T. (2016). Research Performance Based Funding Systems: a Comparative Assessment (JRC Science For Policy Report). European Commission: Joint Research Centre. https://rio.jrc.ec.europa.eu/en/file/9514/download?token=-8JG6aKx. Accessed 25 July 2016.

  • Kronman, U., Gunnarsson, M., & Karlsson, S. (2010). The bibliometric database at the Swedish Research Council—contents, methods and indicators. Vetenskapsrådet. https://doi.org/10.13140/rg.2.1.1271.6244. Accessed 12 Dec 2017.

  • Moed, H. F. (2008). UK Research Assessment Exercises: Informed judgments on research quality or quantity? Scientometrics, 74(1), 153–161.

    Article  Google Scholar 

  • Moed, H. F., Burger, W. J. M., Frankfort, J. G., & Van Raan, A. F. J. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14(3), 131–149.

    Article  Google Scholar 

  • Nelhans, G., & Eklund, P. (2015). Resursfördelningsmodeller. På bibliometrisk grund vid ett urval svenska lärosäten (No. 30) (p. 64). Högskolan i Borås. http://hb.diva-portal.org/smash/get/diva2:789871/FULLTEXT01.pdf.

  • NSD—Norsk senter for forskningsdata. (2017). Kriterier for godkjenning av publiseringskanaler. https://dbh.nsd.uib.no/publiseringskanaler/OmKriterier Accessed 26 Apr 2017.

  • Öhrn, E., & Lundahl, L. (Eds.). (2013). Kön och karriär i akademin: en studie inom det utbildningsvetenskapliga fältet. Göteborg: Acta universitatis Gothoburgensis.

    Google Scholar 

  • Osuna, C., Cruz-Castro, L., & Sanz-Menéndez, L. (2011). Overturning some assumptions about the effects of evaluation systems on publication performance. Scientometrics, 86(3), 575–592.

    Article  Google Scholar 

  • Pajić, D. (2015). Globalization of the social sciences in Eastern Europe: genuine breakthrough or a slippery slope of the research evaluation practice? Scientometrics, 102(3), 2131–2150.

    Article  Google Scholar 

  • Rabovsky, T. M. (2014). Using data to manage for performance at public universities. Public Administration Review, 74(2), 260–272.

    Article  Google Scholar 

  • Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378.

    Article  Google Scholar 

  • Schneider, J. W., Aagaard, K., & Bloch, C. W. (2016). What happens when national research funding is linked to differentiated publication counts? A comparison of the Australian and Norwegian publication-based funding models. Research Evaluation, 25(3), 244–256.

    Article  Google Scholar 

  • Silfver, E. (2013). Subjektspositioner i olika utbildningsvetenskapliga miljöer. In E. Öhrn & L. Lundahl (Eds.), Kön och karriär i akademin: en studie inom det utbildningsvetenskapliga fältet (pp. 73–106). Göteborg: Acta universitatis Gothoburgensis.

    Google Scholar 

  • Sivertsen, G. (2016a). A bibliometric indicator with a balanced representation of all fields. In Proceedings of the 21ST International Conference on Science and Technology Indicators. Valencia, Spain (pp. 910–914).

  • Sivertsen, G. (2016b). Publication-based funding: The Norwegian model. In M. Ochsner, S. E. Hug, & H.-D. Daniel (Eds.), Research assessment in the humanities (pp. 79–90). Cham: Springer.

    Chapter  Google Scholar 

  • Sivertsen, G., & Larsen, B. (2012). Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: an empirical analysis of the potential. Scientometrics, 91(2), 567–575.

    Article  Google Scholar 

  • Svensson, M. (2013). Tekniska system—att se teknik i sammanhang. In J. Hallström & C. Klasander (Eds.), Ginners teknikdidaktiska handbok: några teser om teknik, skola och samhälle | The Ginner Handbook of Technology Education: Some Theses about Technology, School and Society (pp. 73–85). Norrköping: Centrum för tekniken i skolan (CETIS), Linköpings universitet.

    Google Scholar 

  • Thomson Reuters. (2010, April 27). Web of Science Coverage Expansion. https://community.endnote.com/t5/Citation-Impact-Center/Web-of-Science-Coverage-Expansion/ba-p/10663 Accessed 13 Sept 2018.

  • University of Gothenburg & Universitetslednings kansli. Fördellning av tillkomande anslag för forskning 2009, §10 (2008). http://medarbetarportalen.gu.se/digitalAssets/1417/1417612_Fordelning_av_tillkommande_anslag_2009.pdf Accessed 11 Dec 2015.

  • van Leeuwen, T. N., van Wijk, E., & Wouters, P. F. (2016). Bibliometric analysis of output and impact based on CRIS data: a case study on the registered output of a Dutch university. Scientometrics, 106(1), 1–16.

    Article  Google Scholar 

  • Vanderstraeten, R. (2011). Scholarly communication in education journals. Social Science History, 35(1), 109–130.

    Article  Google Scholar 

  • Vanderstraeten, R., Vandermoere, F., & Hermans, M. (2016). Scholarly communication in AERA journals, 1931 to 2014. Review of Research in Education, 40, 38–61.

    Article  Google Scholar 

  • Vetenskapsrådet. (2017). The bibliometric database at the Swedish Research Council—contents, methods and indicators (No. 113-2010–6148). Vetenskapsrådet. https://vr.se/download/18.7a00beb015988a0395140244/1485525234635/The+bibliometric+database+at+the+Swedish+Research+Council+-+contents%2C+methods+and+indicators.pdf. Accessed 13 Jan 2018.

  • Yang Hansen, K., & Gustafsson, J.-E. (2016). Causes of educational segregation in Sweden—school choice or residential segregation. Educational Research and Evaluation, 22(1–2), 23–44.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Tim C.E. Engels, Raf Guns, Sverker Lindblad, Gustaf Nelhans, and Ernst D. Thoutenhoofd, among others, for their critical remarks which have helped to develop the ideas presented here.

Author information

Authors and Affiliations

  1. Centre for R&D Monitoring (ECOOM), Faculty of Social Sciences, University of Antwerp, Middelheimlaan 1, 2020, Antwerp, Belgium

    Linda Sīle

  2. Department of Sociology, Ghent University, Korte Meer 3-5, 9000, Ghent, Belgium

    Raf Vanderstraeten

Authors
  1. Linda Sīle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raf Vanderstraeten
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linda Sīle.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 23 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sīle, L., Vanderstraeten, R. Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014). Scientometrics 118, 71–91 (2019). https://doi.org/10.1007/s11192-018-2963-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-018-2963-8

Keywords

Mathematics Subject Classification

JEL Classification

Search

Navigation