Advertisement

Scientometrics

, Volume 118, Issue 1, pp 71–91 | Cite as

Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014)

  • Linda SīleEmail author
  • Raf Vanderstraeten
Article

Abstract

We present a novel way to frame a discussion of changes in publication patterns that occur in a context of performance-based research funding systems (PRFSs). Adopting an approach derived from social epistemology, we foreground the dialectical nature of knowledge. This allows us to relate changes in publication patterns to PRFS and show the tensions that emerge between relatively diverse research fields with diverse publication practices and bibliometric indicators within PRFSs that reinforce a singular view of research goals. Specifically, we highlight that the employment of bibliometric indicators result in a fixed hierarchy among communication media that may be at odds with the goals within research fields subjected to PRFSs. These ideas are illustrated with an empirical analysis of changes in publication patterns within the field of educational research at the University of Gothenburg (GU; 2005–2014) in Sweden. We contrast the observed changes with implicit priorities in the national and institutional PRFS that operate in this context since 2009. Findings from bibliometric analysis indicate a move away from publication traditions that used to be characteristic of educational research: the growth in the number of journal articles is greater than that in the number of book chapters, while the number of reports is on a declining slope. In relation to PRFSs, we show that conclusive judgement on the desirability of the observed changes is hardly achievable. If one adopts the aims implicit in PRFS, research performance appears to be enhanced. If one sides with the views of many educational researchers, then some of the trends might be an indication of undesirable changes.

Keywords

Publication patterns Bibliometric indicators Research performance Educational research Sweden Social epistemology 

Mathematics Subject Classification

62P25 

JEL Classification

I23 

Notes

Acknowledgements

The authors thank Tim C.E. Engels, Raf Guns, Sverker Lindblad, Gustaf Nelhans, and Ernst D. Thoutenhoofd, among others, for their critical remarks which have helped to develop the ideas presented here.

Supplementary material

11192_2018_2963_MOESM1_ESM.docx (24 kb)
Supplementary material 1 (DOCX 23 kb)

References

  1. Aagaard, K. (2015). How incentives trickle down: Local use of a national bibliometric indicator system. Science and Public Policy, 42(5), 725–737.  https://doi.org/10.1093/scipol/scu087.CrossRefGoogle Scholar
  2. Aagaard, K., Bloch, C., & Schneider, J. W. (2015). Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator. Research Evaluation, 24(2), 106–117.CrossRefGoogle Scholar
  3. Aldberg, H., & Jacobsson, C. (2014). Bibliometriskt underlag för medelsfördelning. Vetenskapsrådet. https://www.vr.se/download/18.5a947f0d145b21c1709f9d/1399897207262/Bibliometriskt+underlag+f%C3%B6r+medelsf%C3%B6rdelning_140512.pdf. Accessed 20 Mar 2016.
  4. Angervall, P., & Gustafsson, J. (2014). Becoming an Academic Researcher. Policy Futures in Education, 12(2), 191–199.CrossRefGoogle Scholar
  5. Anslagsfördelning och kostnadsdebitering för budgetåret 2015 samt planeringsramar för 2016–2017., Pub. L. No. E 2014/257 § Universitetsstyrelsen (2014). http://medarbetarportalen.gu.se/digitalAssets/1497/1497483_anslagsf–rdelning-och-kostnadsdebitering-2015-inkl-bilagor.pdf. Accessed 22 Apr 2016.
  6. Butler, L. (2003). Modifying publication practices in response to funding formulas. Research Evaluation, 12(1), 39–46.CrossRefGoogle Scholar
  7. Butler, L. (2004). What happens when funding is linked to publication counts? In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research: the use of publication and patent statistics in studies of S&T systems (pp. 389–405). Dordrecht: Kluwer Academic Publishers.Google Scholar
  8. Carlsson, H. (2009). Allocation of Research Funds Using Bibliometric Indicators—Asset and Challenge to Swedish Higher Education Sector. InfoTrend, 64(4), 82–88.Google Scholar
  9. de Rijcke, S., Wouters, P. F., Rushforth, A. D., Franssen, T. P., & Hammarfelt, B. (2015). Evaluation practices and effects of indicator use—a literature review. Research, Evaluation, rvv038.Google Scholar
  10. Dees, W. (2008). Innovative scientometric methods for a continuous monitoring of research activities in educational science. In H. Kretschmer & F. Havemann (Eds.), Proceedings of WIS 2008, Berlin (pp. 1–10). Berlin.Google Scholar
  11. Dougherty, K. J., Jones, S. M., Lahr, H., Pheatt, L., Natow, R. S., & Reddy, V. (2016). Performance funding for higher education. Baltimore: Johns Hopkins University Press.Google Scholar
  12. Engels, T. C. E., Ossenblok, T. L. B., & Spruyt, E. H. J. (2012). Changing publication patterns in the Social Sciences and Humanities, 2000–2009. Scientometrics, 93(2), 373–390.CrossRefGoogle Scholar
  13. Englund, T. (2006). New Trends in Swedish Educational Research. Scandinavian Journal of Educational Research, 50(4), 383–396.CrossRefGoogle Scholar
  14. Ett lyft för forskning och innovation, Pub. L. No. Regeringens proposition 2008/09:50 (2008). http://www.regeringen.se/contentassets/05cb6c62a34e4b37a114611a3ebcbd5b/ett-lyft-for-forskning-och-innovation-prop.-20080950. Accessed 10 Jan 2014.
  15. Fuller, S. (2002). Social epistemology (2nd ed.). Bloomington: Indiana University Press.Google Scholar
  16. Gläser, J. (2017). A fight on epistemological quicksand: Comment on the dispute between van den Besselaar et al. and Butler. Journal of Informetrics, 11(3), 927–932.CrossRefGoogle Scholar
  17. Gothenburg University Library: Allocation. (2015). http://www.ub.gu.se/publicera/bibliometri/resursfordelning/ Accessed 4 Dec 2015.
  18. Gothenburg University Library: Publication types in GUP. (2015). http://www.ub.gu.se/publicera/publikationsbas/publtyper/. Accessed 26 Jan 2016.
  19. Hammarfelt, B., & de Rijcke, S. (2015). Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation, 24(1), 63–77.CrossRefGoogle Scholar
  20. Hansen, M., & Lindblad, S. (2010). Forskningskommunikation och Publiceringsmönsrwe inom Utbildningsvetenskap (VETENSKAPSRÅDETS RAPPORTSERIE) (p. 95). Stockholm: Vetenskapsrådet. https://publikationer.vr.se/produkt/forskningskommunikation-och-publiceringsmonster-inom-utbild/ Accessed 14 Oct 2015.
  21. Hicks, D. (2004). The Four Literatures of Social Science. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 473–496). Dordrecht: Kluwer Academic Publishers.Google Scholar
  22. Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251–261.CrossRefGoogle Scholar
  23. Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520, 429–431.CrossRefGoogle Scholar
  24. Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of “new” performance funding in higher education. Educational Evaluation and Policy Analysis, 37(4), 501–519.CrossRefGoogle Scholar
  25. Inriktningsbeslut för budgetarbetet 2012. Sammanfattning., Pub. L. No. V 2011/190 (2011). http://medarbetarportalen.gu.se/digitalAssets/1535/1535388_inriktningsbeslut-f–r-budgetarbetet-2012.pdf. Accessed 4 Apr 2016.
  26. Jiménez-Contreras, E., de Moya Anegón, F., & López-Cózar, E. D. (2003). The evolution of research activity in Spain. Research Policy, 32(1), 123–142.CrossRefGoogle Scholar
  27. Jonkers, K., & Zacharewicz, T. (2016). Research Performance Based Funding Systems: a Comparative Assessment (JRC Science For Policy Report). European Commission: Joint Research Centre. https://rio.jrc.ec.europa.eu/en/file/9514/download?token=-8JG6aKx. Accessed 25 July 2016.
  28. Kronman, U., Gunnarsson, M., & Karlsson, S. (2010). The bibliometric database at the Swedish Research Councilcontents, methods and indicators. Vetenskapsrådet.  https://doi.org/10.13140/rg.2.1.1271.6244. Accessed 12 Dec 2017.
  29. Moed, H. F. (2008). UK Research Assessment Exercises: Informed judgments on research quality or quantity? Scientometrics, 74(1), 153–161.CrossRefGoogle Scholar
  30. Moed, H. F., Burger, W. J. M., Frankfort, J. G., & Van Raan, A. F. J. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14(3), 131–149.CrossRefGoogle Scholar
  31. Nelhans, G., & Eklund, P. (2015). Resursfördelningsmodeller. På bibliometrisk grund vid ett urval svenska lärosäten (No. 30) (p. 64). Högskolan i Borås. http://hb.diva-portal.org/smash/get/diva2:789871/FULLTEXT01.pdf.
  32. NSD—Norsk senter for forskningsdata. (2017). Kriterier for godkjenning av publiseringskanaler. https://dbh.nsd.uib.no/publiseringskanaler/OmKriterier Accessed 26 Apr 2017.
  33. Öhrn, E., & Lundahl, L. (Eds.). (2013). Kön och karriär i akademin: en studie inom det utbildningsvetenskapliga fältet. Göteborg: Acta universitatis Gothoburgensis.Google Scholar
  34. Osuna, C., Cruz-Castro, L., & Sanz-Menéndez, L. (2011). Overturning some assumptions about the effects of evaluation systems on publication performance. Scientometrics, 86(3), 575–592.CrossRefGoogle Scholar
  35. Pajić, D. (2015). Globalization of the social sciences in Eastern Europe: genuine breakthrough or a slippery slope of the research evaluation practice? Scientometrics, 102(3), 2131–2150.CrossRefGoogle Scholar
  36. Rabovsky, T. M. (2014). Using data to manage for performance at public universities. Public Administration Review, 74(2), 260–272.CrossRefGoogle Scholar
  37. Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378.CrossRefGoogle Scholar
  38. Schneider, J. W., Aagaard, K., & Bloch, C. W. (2016). What happens when national research funding is linked to differentiated publication counts? A comparison of the Australian and Norwegian publication-based funding models. Research Evaluation, 25(3), 244–256.CrossRefGoogle Scholar
  39. Silfver, E. (2013). Subjektspositioner i olika utbildningsvetenskapliga miljöer. In E. Öhrn & L. Lundahl (Eds.), Kön och karriär i akademin: en studie inom det utbildningsvetenskapliga fältet (pp. 73–106). Göteborg: Acta universitatis Gothoburgensis.Google Scholar
  40. Sivertsen, G. (2016a). A bibliometric indicator with a balanced representation of all fields. In Proceedings of the 21ST International Conference on Science and Technology Indicators. Valencia, Spain (pp. 910–914).Google Scholar
  41. Sivertsen, G. (2016b). Publication-based funding: The Norwegian model. In M. Ochsner, S. E. Hug, & H.-D. Daniel (Eds.), Research assessment in the humanities (pp. 79–90). Cham: Springer.CrossRefGoogle Scholar
  42. Sivertsen, G., & Larsen, B. (2012). Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: an empirical analysis of the potential. Scientometrics, 91(2), 567–575.CrossRefGoogle Scholar
  43. Svensson, M. (2013). Tekniska system—att se teknik i sammanhang. In J. Hallström & C. Klasander (Eds.), Ginners teknikdidaktiska handbok: några teser om teknik, skola och samhälle | The Ginner Handbook of Technology Education: Some Theses about Technology, School and Society (pp. 73–85). Norrköping: Centrum för tekniken i skolan (CETIS), Linköpings universitet.Google Scholar
  44. Thomson Reuters. (2010, April 27). Web of Science Coverage Expansion. https://community.endnote.com/t5/Citation-Impact-Center/Web-of-Science-Coverage-Expansion/ba-p/10663 Accessed 13 Sept 2018.
  45. University of Gothenburg & Universitetslednings kansli. Fördellning av tillkomande anslag för forskning 2009, §10 (2008). http://medarbetarportalen.gu.se/digitalAssets/1417/1417612_Fordelning_av_tillkommande_anslag_2009.pdf Accessed 11 Dec 2015.
  46. van Leeuwen, T. N., van Wijk, E., & Wouters, P. F. (2016). Bibliometric analysis of output and impact based on CRIS data: a case study on the registered output of a Dutch university. Scientometrics, 106(1), 1–16.CrossRefGoogle Scholar
  47. Vanderstraeten, R. (2011). Scholarly communication in education journals. Social Science History, 35(1), 109–130.CrossRefGoogle Scholar
  48. Vanderstraeten, R., Vandermoere, F., & Hermans, M. (2016). Scholarly communication in AERA journals, 1931 to 2014. Review of Research in Education, 40, 38–61.CrossRefGoogle Scholar
  49. Vetenskapsrådet. (2017). The bibliometric database at the Swedish Research Councilcontents, methods and indicators (No. 113-2010–6148). Vetenskapsrådet. https://vr.se/download/18.7a00beb015988a0395140244/1485525234635/The+bibliometric+database+at+the+Swedish+Research+Council+-+contents%2C+methods+and+indicators.pdf. Accessed 13 Jan 2018.
  50. Yang Hansen, K., & Gustafsson, J.-E. (2016). Causes of educational segregation in Sweden—school choice or residential segregation. Educational Research and Evaluation, 22(1–2), 23–44.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.Centre for R&D Monitoring (ECOOM), Faculty of Social SciencesUniversity of AntwerpAntwerpBelgium
  2. 2.Department of SociologyGhent UniversityGhentBelgium

Personalised recommendations