Introduction and background

Over the last couple of decades, European Academia has experienced significant changes (Capano 2011; Olssen and Peters 2005). Largely influenced by New Public Management (NPM), new standards for quality assessment and control were developed (Lorenz 2012; Leišytė 2016). Many countries have implemented national performance-based university research funding systems (PRFS), making government funding of universities dependent on ex post evaluations of research output (Hicks 2012). The properly best-known example is the British Research Assessment Exercises (RAE) introduced in 1986 (now the Research Excellence Framework (REF) (see e.g. Martin 2011; Barker 2007)). By 2014, 15 of 28 member states of the European Union applied a PRFS (Jonkers and Zacharewicz 2016). But also, outside Europe this type of systems is used. In particular, the (now former) Australian system has received attention (see Butler 2003). Moreover, research productivity enters into performance-based funding models in some U.S. states (see Rabovsky 2012, 2014). The purpose of the PRFS is to boost research quality and quantity (Hicks 2009). As well as to enhance the accountability, effectiveness and legitimacy of public spending on research (Whitley and Gläser 2007).

Most of the PRFS are designed to allocate funding at aggregated levels, but the incentives tend to “trickle down” (Aagaard 2015; see also Krog Lind 2019) and several studies show how they are used at the individual level as well, and that they influence the publication behavior of researchers, especially those from the Social Sciences and Humanities (SSH) (Bloch and Schneider 2016; Mouritzen and Opstrup 2019; Haddow and Hammarfelt 2018, 2019). Thus, studies show how researchers may adapt their publication behavior to how they are measured. Dahler-Larsen (2013) coined the expression “constitutive effects” to describe changes influenced by measurement. Espeland and Sauder (2007) defines it as “reactivity” when people alter their behavior in reaction to being evaluated, observed, or measured.

In Denmark, the Danish Bibliometric Research Indicator (BFI) was introduced in 2009 as a result of negotiations between the Ministry of Science and Higher Education and the Universities Denmark (the association of Danish universities). The introduction of the BFI followed the international trend of using New Public Management in higher education and it was decided to develop a research indicator based on the Norwegian model (Mouritzen and Opstrup 2019; Aagaard 2018).

Denmark has been described as one of the ‘reluctant reformers’ in terms of implementing New Public Management reforms in the public sector. But in fact, Denmark went from being somewhat hesitant in terms of political intervention in the internal management structures of the higher education institutions to becoming a frontrunner (Degn 2014; Pinheiro and Stensaker 2014).

The major shift came in 2003, when the new university act was implemented.Footnote 1 Due to this act, the Danish universities underwent a series of significant changes that shifted the governance paradigm from “Professional rule” towards “New Public Management” (NPM) (Torfing et al. 2020).

Professional rule, as a governance paradigm, implies that decision-making power is delegated to the relevant profession(s) and professionals. Given professions’ specialized knowledge, only the professionals are expected to possess the information and expert insight necessary to be responsible for the governance of the relevant occupational area and management of the given occupational group. Governance should, in this view, be based on the professional norms set and sanctioned by the profession (Torfing et al. 2020: 43).

Historically, academia has been an archetype for Professional rule. To paraphrase Polanyi (1962), in the “Republic of Science” authority is established between scientists, not above them. Scientists exercise their authority over each other: Research and researcher quality are assessed by peer-review processes and assessment committees consisting of members from the discipline; leadership and decision-making are based on collegial decisions; universities and academics enjoy institutional autonomy and academic freedom (Bleiklie and Kogan 2007). Thus, academia is, in other words, a strong profession with a high degree of professional autonomy and well-established professional norms and its own “incentive systems”.

NPM, however, limits the self-regulating power of professions and professionals (Lorenz 2014). NPM sets focus on (external) accountability, efficiency, “value for money” etc. The intellectual source of inspiration stem from neoclassical economics (homo economicus). Agents (institutional and/ or individual) are assumed to be rational, self-interested and utility maximizing and must, therefore, be controlled and incentivized to perform in accordance with the interests of the principal (on principal-agent problems, see e.g. Miller 2005). The formula for this, canonized by NPM, is to introduce (quasi-marked) competition supplemented with (quantified) performance measures linked to conditional (financial) incentives (Torfing et al. 2020: 56).

The BFI (and PRFS, in general) in many ways reflects this line of thought (Mouritzen and Opstrup 2019; Hicks 2012). At the same time, however, Professional rule continues to play an important role in academia—which is also reflected in the organization and management of the BFI system (see Mouritzen and Opstrup 2019, ch.1 for a presentation of the background and institutional setup of the system).

Most important to our study are the so-called BFI lists, maintained by 67 disciplinary committees. The list itself consist of two preferred lists of publication outlets: One list of journals and one list of publishers. The lists are dynamic and includes 20.433 journals as of 2019 divided into three tiers while the list of publishers includes 1.163 publishers as of 2019 and has only two tiers.

All peer reviewed publications in journals or with publishers on the BFI list generates a certain number of points based on the publication type and the rank level of the publication outlet (Table 1). In addition, a fixed number of points are awarded to patents and doctoral dissertations (a publication type distinct from a PhD dissertation, a doctoral dissertation is written to obtain the Danish ‘doktorgrad’, e.g. dr.phil. It is typically written by researchers that have had a lengthy and distinguished academic career). We note that the actual points allocation is significantly more complicated, as points are fractionalized among co-authorships at eligible Danish institutes (Wien et al. 2017). Every peer reviewed Danish journal article published in any of the journals on the BFI list will earn 1, 3 or 5 points depending on the level of the journal. Allocation of points to research outputs published in the list of publishers is slightly more complicated since the amount of points earned also depends on the type of publication (for example an anthology in a BFI1 publisher will only generate 0.5 points, while a monograph published in a BFI2 publisher will generate 8 points).

Table 1 Simplified distribution of the points awarded to the publication types measured by the bibliometric research indicator (BFI) as of 2019, per BFI level.
Full size table

The BFI system was implemented in 2009, while the performance-based distribution of the national block grant to Danish universities started in 2010 (Table 2). To distribute funds, first the publications of year x are harvested. Then the indicator statistics (how many points each publication outlet is worth) in year x + 1 are used to award points. Finally, the points are used to allocate funds from the Block Grant in year x + 2. In the beginning, the financial incentive was rather weak. In 2010, one BFI point was approx. worth €200 for the publishing university. However, by 2014 the value had increased tenfold, one BFI point being worth approx. €2000 (Mouritzen and Opstrup 2019: 15–15) (see also Table 2).

Table 2 Overview of the BFI system since its implementation in 2008 until 2020.
Full size table

While the BFI lists for journals and publishers are updated almost every year, the BFI system has been updated twice since its implementation. The first update in 2013 introduced leveling to the BFI list for publishers, splitting the publishers into two levels. It also introduced substantial changes to the publication types: (1) The publication type “conference contributions” was established, removing them from either the journal article or the anthology contributions publication types, and (2) Book series were recognized as a separate type of publication outlet, splitting the monograph and anthology contribution publication types in two. The book series were added to the BFI list for journals. By comparison, the second update in 2018 was a modest one, and only introduced a third level to the BFI list for journals.

The BFI lists were originally generated, and are currently being maintained, by professors from the disciplines. The lists for both journals and publishers are updated almost every year and are reviewed by 67 disciplinary committees. They assess researcher requests for new inclusions to the lists and continued relevance of titles already on the lists. Any disputes among the 67 disciplinary committees are settled by an Academic Commission consisting of one representative from each of the 5 main disciplines (SSH and STEM).

There is a considerable lag time in the system. The expert panels typically meet 1–2 times per year to discuss changes, they put the proposed changes forward to the BFI-secretariat at the Danish Ministry for Science and Higher Education who maintain the lists (https://ufm.dk/en/research-and-innovation/statistics-and-analyses/bibliometric-research-indicator/governance, accessed Oct. 4th 2020). The BFI lists are then updated for the following year. A summary of all the changes to the BFI list for journals is presented in table S1. The changes to the BFI list for publishers are given in table S2 since the introduction of leveling to the publisher list in the 2013 update to the BFI system.

There can also be considerable time lag between conceptualization of a research output and actual publication. E.g. the time to publication in Social Sciences and Humanities (SSH) can be lengthy, thus it stands to reason that we still see changes in publication patterns in 2013 as result of the introduction of economy to the BFI system in 2010.

The number of journals on the BFI lists varies considerably from overall discipline to discipline (table S3), where we see a range from about 1000 to over 4000 journals (plus a smaller number of broad subject journals). Though the majority of the journals on the list do not change from year to year, some change does happen on a yearly basis; some years considerably more than others (e.g. 2017 compared to 2019).

While research into the effects of the BFI has indicated that its implementation was not limited to management (Mouritzen et al. 2018: 285). Since allocation of funds is partly dependent on how many BFI points the universities earn, incentives have been created for management to encourage their units to produce points. It is not uncommon for Heads of Departments to instruct the researchers in their faculty to “earn at least 2 BFI points annually” or provide financial rewards for publishing in certain journals. This tendency means that the BFI becomes defining for a researcher’s publication behaviors and may in fact lead to changes.

The aim of this paper is to analyze changes in the publication patterns of Danish researchers since the introduction of the BFI by examining the total number of Danish research publications awarded BFI points per year, per publication type, and per research discipline. The rationale for this is threefold:

Firstly, the two potentially contradicting governance paradigms co-existing or competing in Danish academia is bound to create dilemmas: If the principles of Professional rule are most predominant, scholars’ decisions about research and publication is likely to be influenced by what is recognized by peers as ‘right’ and ‘wrong’ or ‘good’ and ‘bad’ and we should not expect publication patterns to necessarily change following the introduction of the BFI. On the other hand, the BFI is, at least to some degree, an externally imposed set of authoritative criteria for what is to be considered ‘good’ publication behavior, which it attempts to stimulate via financial incentives at the university level. In its essence, the BFI system defines what count as scientific publications and are eligible for BFI points, which outlets are considered ‘legitimate’ publication channels, and the ‘value’ of different types of publications (Mouritzen and Opstrup 2019: 61). Thus, if publications patterns have changed significantly, this may indicate that researchers optimize because of the BFI list.

Secondly, research into the effects of the BFI indicates that the effects of its implementation vary across the different disciplines (Mouritzen et al. 2018: 279). It appears that effects have been relatively modest in STEM (science technology engineering medicine), while more severe within SSH (Social Sciences and Humanities). The reason for this, according to previous researchers, is that STEM has been focusing and optimizing against a completely different set of performance indicators, e.g. the h-index, and targeting publication channels with high Journal Impact Factors (JIFs) for many years. To SSH, Performance Based Measures were new in 2010 when the BFI was established and consequently received much more attention (Mouritzen et al. 2018: 279). In this study, we intend to document and provide more weight to this hypothesis by using new data sources.

Thirdly and finally, the BFI is currently being revised. Tracing any potential malfunction of the current system may be useful for the development of the new BFI system.

The research questions (RQs) are the following:

  1. 1.

    To what extent do Danish researchers increasingly get published in the higher tier of the BFI (BFI2-3) and are there differences between the disciplines?

  2. 2.

    To what extent have Danish researchers changed their choice of publication type after the introduction of the BFI across the disciplines?

Methods and data

Every university in Denmark prepares a report for the Danish Ministry of Science and Higher Education of the number of peer reviewed publications of each type they have produced in that year. The ministry then awards points according to the plan shown in Table 1 and fractionalizes for co-authorships. This publicly available database provides a unique and high quality overview of the research produced in Denmark, as opposed to the more traditional sources for publication data analysis like Web of Science or Scopus which have less complete coverage of publications in languages other than English, certain fields within the Social Sciences and Humanities and non-journal article publications.

The total number of research publications produced in Denmark from 2009 to 2019 is publicly available on the BFI registry website: https://bfi.fi.dk/Publication/NationalAnalysis. This data was collected on Sep. 28th, 2020 and processed in Python and MS Excel. The number of publications is split by publication type, outlet and research discipline. The ministry distinguishes 4 general research disciplines as follows: Health Sciences, Natural Sciences & Technology, Humanities, and Social Sciences. And 6 general publication types as follows: journal articles, monographs, anthology contributions, conference contributions, patents and doctoral dissertations (distinct from a PhD dissertation, a doctoral dissertation is written to obtain the Danish ‘doktorgrad’, e.g. dr.phil).

In our analyses we examined the total number of published and BFI point awarded research publications produced by researchers in Denmark in the 4 general research disciplines. To examine whether the publication preferences of Danish researchers are shifting due to targeting of BFI2-3 publications outlets (RQ1), we focus primarily on the years 2013–2019 as the leveling of publishers was only introduced in 2012. The unit of analysis is the number of research publications. Journals have been assigned levels for the full span of the BFI system, so analysis of the number of articles per BFI level is provided from 2009 to 2019.

To examine whether Danish researchers are targeting different publication types (RQ2), we used the publicly available data on the number of publications, per publication type and research discipline from 2009 to 2019. The unit of analysis is the number of research publications.

The BFI lists for publishers and journals are also publicly available on the Danish Ministry of Science and Higher Education’s website from 2008/2009 to 2019 (https://ufm.dk/forskning-og-innovation/statistik-og-analyser/den-bibliometriske-forskningsindikator/BFI-lister).This data was collected on Sept. 28th 2020 and processed in Python and MS Excel. The processed data is presented in Table 2, S1 and S2 on the BFI lists of publishers and journals. The data on available funding was sourced from Aagaard 2016.

To track changes in the definition of the BFI levels, we examined the yearly definition of the BFI list for journals and publishers eligible for BFI points. This data is publicly available from before the inception of the BFI in 2008 until 2019, here: https://ufm.dk/forskning-og-innovation/statistik-og-analyser/den-bibliometriske-forskningsindikator/BFI-lister/historiske-bfi-lister/historiske-bfi-lister. This data was collected on Sep. 28th 2020 and processed in Python and MS Excel.

RQ 1: Increased focus on BFI2-3

The publication productivity of all Danish universities is presented in Fig. 1, from 2013 to 2019, for research published in BFI1 and BFI2-3 journals and publishers. To facilitate comparisons, we have aggregated BFI level 2 and 3 as BFI3 has only existed since 2017. The publication productivity is also given per research discipline, defined by the Danish Ministry of Science and Higher Education as follows: Health Sciences, Natural Sciences & Technology (abbreviated here as Nat Sci & Tech), Humanities, and Social Sciences.

Fig. 1
figure 1

Source: BFI registry

Overview of all BFI awarded publications in Denmark from 2013 to 2019, per research discipline. a Total number of all publications in BFI1 journals and publishers. b Total number of all publications in BFI2-3 journals and publishers. c Share of publications in BFI1 journals and publishers. d Share of publications in BFI2-3 journals and publishers. Research disciplines are divided into four categories, in accordance with the Danish ministry for higher education and science’s definition, as: Health Sciences, Natural Sciences & Technology (here Nat Sci & Tech), Humanities and Social Sciences. The dotted lines are a guide for the eye.

Full size image

It is clear that the overall number of publications has seen a steady increase both in BFI1 and BFI2-3 (Fig. 1a, b). The STEM fields are the most productive, accounting for a total of 13.317 BFI1 and 7.000 BFI2-3 publications in 2019, while SSH produced 3.686 BFI1 and 2.634 BFI2-3 publications in that time.

The trends become more interesting when we examine the share of publications that were published in either BFI1 or BFI2-3 (Fig. 1c, d). From 2013 to 2019 the share of publications in BFI2-3 experienced a modest 2% increase: In 2013, 34% of all publications were published in BFI2-3 and 66% were published in BFI1, while in 2019 36% of all publications were published in BFI2-3 and 64% were published in BFI2-3. Based on these figures alone we cannot argue for any overall significant change in researcher’s publication preference between BFI1 and BFI2-3. Yet, if we examine the research disciplines individually the changes appear to be more pronounced.

The increase in BFI2-3 publications is largest in the Social Sciences, ballooning by 10%, from a 37 to a 47% share. Followed by the Humanities, which sees an increase in the share of BFI2-3 publications by 5%, from 31 to 36%. Turning to the STEM fields: Natural Sciences & Technology experience a modest 3% increase, while the Health Sciences actually see a 2% decrease in BFI2-3 publications.

Examining only the overall publication trends obscures differences in the targeting of specific publication types. As journal articles account for the overwhelming majority of publications overall (see the distribution of publication types per research discipline in the Electronic Supplementary Material, figure s1), the share of journal articles published in BFI2-3 is presented in Fig. 2a. The leveled data for journal articles is available for a longer time period, going back to the introduction of the BFI.

Fig. 2
figure 2

Source: BFI registry

Share of journal articles, monographs and anthologies published in BFI2-3 journals and publishers, per research discipline. a Share of journal articles published in BFI2-3 journals from 2009 to 2019. b Share of monographs and anthologies in BFI2-3 publishers from 2013 to 2019. c Share of journals on the BFI list with a level of 2 or 3 from 2008 to 2019. d Share of publishers on the BFI list with a level of 2 or 3 from 2012 to 2019. All figures are plotted at the same y-scale. Research disciplines are divided into four categories, in accordance with the Danish ministry for higher education and science’s definition, as: Health Sciences, Natural Sciences & Technology (here Nat Sci & Tech), Humanities and Social Sciences. The dotted lines are a guide for the eye.

Full size image

While there appears to be little change in the overall share of BFI2-3 journal articles, here it becomes clear that the Natural Sciences & Technology were a first mover, increasing focus on the BFI2-3 journals that would reap more BFI points by 10%, from 2009 to 2019. Social Sciences appears to follow their lead just a few years later, directly increasing their share of BFI2-3 journal articles from 2012. Interestingly, the share of BFI2-3 journal articles in the Humanities decreased from 2009 to 2011, after which they too appear to shift their focus on the higher tiered BFI2-3 journals. Health Sciences appears relatively unaffected by the introduction of the BFI list, experiencing no change in the share of BFI2-3 journal articles. As of 2019, both Natural Sciences & Technology and Social Sciences publish half of their journal articles in BFI2-3 journals, at 48 and 50% respectively.

The BFI system has recognized conference contributions in their own distinct tiered publication type since 2013. These consist of articles in conference proceedings and abstracts and are primarily published by the Natural Sciences & Technologies. For the sake of completeness, the share of conference contributions is included in the Electronic Supplementary Material (figure S2). Yet, there is almost no change in the overall share of BFI2-3 publication in research articles from 2013 to 2019 except that the Health Sciences appear to have abandoned them as a publication type. Even in the Natural Sciences & Technologies, the increase in BFI2-3 research articles is only 2%.

Monographs and anthologies account for about half of the total publication output in SSH (figure S1), so if there is a focus on BFI2-3 in SSH we would expect to see it here as well. The share of BFI2-3 monographs and anthologies from 2013 to 2019 is presented in Fig. 2b. Unsurprisingly, SSH is indeed driving the focus on BFI2-3 here, where the share in Social Sciences and Humanities increases by 8 and 5% respectively. In Health Sciences and Natural Sciences & Technology there is a downward trend, decreasing their share by 7 and 2%. We note that these publication types are of less importance to STEM, and that the absolute number of publications is almost insignificant when compared to the journal article output.

The number of publication outlets included in the BFI list changes year to year, and it could be that the increases in the share of BFI2-3 publications could be ascribed simply to a greater proportion of BFI2-3 publication outlets. The yearly share of BFI2-3 publication outlets is presented for journals in Fig. 2c, per research discipline, and for publishers in Fig. 2d. The overall share of BFI2-3 journals has increased since the introduction of the BFI from 12.8% in 2009 to 16.1% in 2019 (Fig. 2c). This pales in comparison to the almost 10% increase in BFI2-3 journal publications seen within that same time period for Nat Sci & Tech and Social Sciences publications. SSH researchers do have a greater share of BFI2-3 journals to choose from than STEM researchers. Turning to publishers as a publication outlet in Fig. 2d, there appears to be almost no change to the share of BFI2-3 publishers (we note that the list of publishers is not divided into research disciplines). Since leveling of publishers was introduced in 2012 there has even been a 0.5% drop in the share of BFI2-3 publishers, even though during that time the share of BFI2-3 monographs and anthologies has increased by 4%. Changes to the BFI list alone do not appear to be able to explain the great gains seen in the share of BFI2-3 publications.

The data therefore supports the notion that publication patterns in the Health Sciences, have remained relatively unaffected by the introduction of the BFI, while in the Natural Sciences & Technology researchers do seem to have increased focus on BFI2-3, as their publishing activities in BFI2-3 journals grew by 10% since the introduction of the BFI list. The data on publication patterns in SSH seems to indicate a significant focus on BFI2-3 across all publication types. The answer to RQ1 is therefore that SSH and the Natural Sciences & Technology appear to have increased focus on publishing in BFI2-3, and we argue that this indicates a change in behavior in order to “earn BFI points”, while the Health Sciences appears not to have changed behavior. However, we are fully aware that based on these data we cannot establish cause and effect. The changes we observe may have other causes than the introduction of the BFI and may merely coincide with its introduction.

RQ 2: Changes in publication types

The second RQ relates to changes in publication types. The question is to what extent have Danish researchers changed their choice of publication type after the introduction of the BFI across the disciplines? Thus, this study examines changes in the number of research outputs produced for each publication type eligible for BFI points (i.e. journal articles, monographs and anthologies, conference contributions, patents and doctoral dissertations) that can be traced over the investigated period.

The total research publication output of Danish universities from 2009 to 2019 is presented per publication type in Fig. 3. At present, journal articles dominate Danish research, accounting for over three quarters of all published research outputs in 2019. From 2009 onwards there is a steady increase in the amount of journal articles, increasing from 12.773 articles in 2009 to 20.643 in 2019. The productivity of all other research publication types appears relatively unchanged. The share of journal articles increased by about 2% in this time, to the detriment of monographs and anthologies which see their share decrease by 2% (Fig. 3b). This could imply that the introduction of the BFI has increased journal article productivity. However, research in authorship patterns has revealed that a steady increase in the number of authors per journal article accounts for much of what appears to be increases in the individual researcher productivity (Fanelli and Larivière 2016). Furthermore, focusing on publication type alone obscures differences in publication behavior between the different research disciplines (Ossenblok et al. 2012).

Fig. 3
figure 3

Source: BFI registry

Overview of all BFI awarded publications in Denmark from 2009–2019, per publication type. a Absolute number of publications per type. b Share of publications per type. The publication types are divided into five categories, in accordance with the Danish Ministry of Science and Higher Education’s definition, as: Journal articles, Monographs and anthologies, Conference contributions, Patents and Doctoral dissertations (distinct from a PhD dissertation, this publication type is written to obtain the Danish ‘doktorgrad’, e.g. dr.phil). We note that the conference contribution category was only introduced in the first update to the BFI in 2013. The dotted lines are a guide for the eye.

Full size image

We note that the publication types cannot directly be compared from before 2012 and after due to changes in the BFI definition of these types introduced in the 2013 BFI update (Table 2). Conference contributions were removed from the journal article and anthology publication types and placed into their own publication type.

Digging deeper into these publication types, the share of each research discipline’s total publication output, per publication type, is presented in Fig. 4 from 2009 to 2019. First, we note that Health Sciences researchers (Fig. 4a) publish their research almost exclusively as journal articles, publishing 96.5% of their research as journal articles, only 2% in monographs and anthologies, and 1% in conference contributions in 2019. While there is a slight 1.5% increase in their preference for journal articles since 2013 (and a 4.3% increase since 2009), we cannot argue for any significant change in the researchers’ publication type preferences at Health Sciences.

Fig. 4
figure 4

Source: BFI registry

The relative share of each publication type of all Danish research publications awarded BFI points from 2009 to 2019, per research discipline. a Health Sciences. b Natural Sciences & Technology. c Humanities. d Social Sciences. The publication types are divided into five categories, in accordance with the Danish Ministry of Science and Higher Education’s definition, as: Journal articles, Monographs and anthologies, Conference contributions, Patents and Doctoral dissertations (distinct from a PhD dissertation, this publication type is written to obtain the Danish ‘doktorgrad’, e.g. dr.phil). The dotted lines are a guide for the eye.

Full size image

The publication type preferences of Natural Sciences & Technology researchers appear almost equally unaffected (Fig. 4b). The share of journal articles has increased by 1.6% from 74.7% in 2013 to 76.3% in 2019, primarily to the detriment of monographs and anthologies which saw a 1.3% decrease in that time. Conference contributions do indeed represent an important publication type for Natural Sciences & Technology researchers, making up around 20% of their publications since the BFI category was established in 2013. While they have clearly focused on BFI2-3 journals for their journal articles, this focus does not appear to have significantly swayed their preference for this publication type over the others.

Turning to SSH (Fig. 4c, d), in the Humanities a preference for journal articles from 2011 to 2013 appears to flip to a preference for publishing monographs and anthologies. The gap between these two publication types widens and narrows over time, yet in 2019 they continue to publish more of their output as monographs and anthologies (49%) than in journal articles (45%). Interestingly, Social Sciences researchers appear to have decreased their focus on monographs and anthologies by 13% from 2009 to 2019 and increased their share of journal articles by 10%.

Anthology contributions are only awarded 0.5 points at BFI1 and 2 points at BFI2, while monographs are “worth” significantly more (5 points at BF1 and 8 points at BFI2-3, see Fig. 1). To uncover differences in researcher preferences for these publication types in SSH, the total number of anthology and monograph publications are presented in Fig. 5. Here, it becomes clear that while monographs are worth more points, the publication rate seems unaffected after the introduction of the BFI system. It is almost constant in the Humanities (Fig. 5a) and experiences only minor fluctuations in the Social Sciences (5b). The increased share of monographs and anthologies in the Humanities seen in Fig. 4c can be entirely attributed to the increase in publication of anthologies.

Fig. 5
figure 5

Source: BFI registry

Number of monographs and anthologies published in SSH from 2013 to 2019. a Humanities. b Social Sciences. The publication types are divided into four categories, in accordance with the Danish Ministry of Science and Higher Education’s definition, as: Monographs in publishers, Monographs in book series, Anthologies in publishers and Anthologies in book series. The solid lines are a guide for the eye.

Full size image

Researchers in the Social Sciences appear to be moving away from the anthology publication type, as the publication rate decreases from a combined high of 1.140 publications in 2014 to only 944 publications in 2019 (17% decrease). This may well be motivated by the points anthologies are worth (BFI1 = 0.5 and BFI2 = 2). Journal articles are awarded 1 point at BFI1, 3 at BFI2 and 5 at BFI3 (Table 1), and it is possible that the increase in journal article publication in the Social Sciences, evident from Fig. 4d, could be influenced by the higher point reward.

The answer to RQ2 is therefore that SSH researchers appear to have changed publication behaviors, focusing more on journal articles in the Social Sciences and more on anthologies in the Humanities. STEM appears not to have changed publication behaviors.

Discussion and conclusions

In this study, we investigated how the BFI may have changed the publication patterns of Danish researchers. Our results show that SSH and Natural Sciences & Technology researchers have increased focus on publishing in BFI2-3, while Health Sciences appears not to have changed behavior (RQ1). We note that these findings are similar to findings on PRFS systems in Norway and Finland (Aagaard et al. 2015).

Digging slightly deeper into the data we find a steep increase in the number of high tier publications for anthologies and monographs—among the preferred types for SSH—while high tier journal articles increase across all disciplines except for Health Science. There is naturally a considerable lag time between conceptualization of a research output and actual publication, and we note that there could be some delay between the submission of a research output to a journal of a specific tier and its acceptance. We must, however, stipulate that our findings are based on published papers and not submitted papers. We do not think this affects our results since it is reasonable to assume that papers are firstly submitted to high tier journals and, if rejected, then submitted to lower tier journals. Therefore, the increase in higher tier publications seems real.

Focusing exclusively on the publication type, it appears that SSH researchers have altered their publication preferences, producing more journal articles in the Social Sciences and more anthologies in the Humanities, while STEM appears not to have changed behavior (RQ2). One factor that may affect our results here is that SSH PhD-theses are increasingly becoming article-based and as such contribute to boosting the number of scientific articles produced within these fields. However, since the production of PhD-theses within SSH in Denmark is relatively modest (about 350 per year, source: forskningsdatabasen.dk), this change of behavior appears to be only “a drop in the ocean”.

Health Sciences’ overall lack of a change in behavior may well stem from their perception that they were already well measured by the new BFI system. There was already a tradition to focus on the absolute number of publications as well as journals with a high JIF.

Shifts in the governance of universities in recent years have “caught” European Academia somewhere in between the “Republic of Science” and the generic “one-size-fits-all” ideas of NPM (Osterloh and Frey 2014). The strong Professional rule of the past is challenged by the introduction of NPM inspired (quantified) quality assessment and control measures (Lorenz 2014). The introduction of PRFS in many countries, including the Danish BFI, are clear examples of this.

The BFI was never intended as a tool for measuring the performance of the individual researchers, nor as an individualized incentive. The findings, however, indicate that the introduction of the BFI nevertheless may have had “constitutive effects” and caused “reactivity” among the scholars within the Natural Sciences & Technology and SSH. We, therefore, conclude that SSH researchers’ publication patterns have changed by displaying increased focus on outlets and publication types that they perceive will give the highest amount of points. Secondly, we conclude that Natural Science & Technology researchers publication patterns have changed by displaying increased focus on higher tiered journals to generate more points.

NPM inspired reforms have been widely criticized for turning universities to the direction of academic capitalism and creating so-called “McUniversities” that mass produce increasingly standardized products (Lorenz 2012; Slaughter and Rhoades 2004). Numerous voices within the research community have raised concerns about whether (quantified) performance measures can be used to make judgments about what constitutes good research, and whether it at all is possible to use (financial) incentives to encourage this (Lewis 2014). Critics accuse evaluations of research output of impairing the intrinsic motivation of scholars by substituting “a taste for science” with “a taste for publications” (Osterloh 2010) and causing a range of undesirable behavioral reactions such as “slicing strategies” (where researchers increase their publication count by dividing their articles into the maximum number of publishable units (Weingart 2005)), the “prostitution of ideas” in order to getting published (Frey 2003), and a homogenization of knowledge production which discourages creative, unorthodox and idiosyncratic research (Gillies 2008). As Butler (2010) summarizes, a general concern raised about PRFS is that such systems favor “mainstream”, disciplinary-based, basic and “safe” research at the expense of applied, interdisciplinary or speculative research.

It is beyond the scope of our study to conclude whether the type and content of the conducted research has changed following the introduction of the BFI. Other studies, however, show that some of the often listed “fears” may be justified while others cannot be substantiated in the case of the Danish BFI (see Mouritzen and Opstrup 2019: 258ff).

Nevertheless, we do believe that our results show that Danish researchers have become caught “between a rock and a hard place” due to the increased focus on PRFS. On the one hand, they may optimize against the simple measures presented, like BFI for SSH and h-index or JIF for STEM. While on the other hand, they face assessment committees using both qualitative and quantitative means of assessing research when applying for tenure and funding. Being measured against two fundamentally different systems occasionally creating opposing incentives for researchers (Wien et al. 2017; Deutz et al. 2020).

If the BFI had been used only as a tool for distribution of funds, and only at the managerial level, as was the original intention of it, it would not have had the potential of changing publication patterns among researchers. However, being a tool for distribution of funds it naturally also creates incentives for the university management in the first order, and then for researchers themselves in the second order. The incentives tend to “trickle down” to the individual level (Aagaard 2015; Mouritzen and Opstrup 2019, ch. 3) and PRFS as the BFI cannot function as “neutral” tools to measure the quality of research but will cause “reactivity” among those measured.

A revised version of the BFI should take this into account.