Introduction

The COVID-19 pandemic is a major driver behind the strong increase in the posting of preprints in the life sciences (Fraser et al., 2021; Puebla et al., 2022)Footnote 1; although it has also been shown that the COVID-19 pandemic had a major negative impact on the amount of time researchers from bench sciences, such as biochemistry, biological sciences, chemistry and chemical enigneering, could spend on research (Myers et al., 2020). However, in the disciplines from the life sciences—such as medicine as well as partly biology and chemistry—in particular, there was a lot of pandemic related research that scientists wanted to disseminate immediately to swiftly expand knowledge about the impact of the virus or vaccine development (Waltman et al., 2021). Rapid dissemination of scientific articles has clear advantages, especially in times of a pandemic—given the sometimes lengthy publication timelines (Fraser et al., 2020). The publication of preprints on repositories has been recognized as an upstream step before official publication in a journal to increase the visibility of research results and speed up dissemination. Estimations expect the amount of preprints to double in less than 10 years (Xie et al., 2021). This has been facilitated, amongst others, by the establishment of preprint repositories such as bioRxiv, medRxiv, and Research Square, which were initially slow to be adopted by the life science community. This changed, however, in April 2020 at the onset of the pandemic (Fraser et al., 2021). This is notable because there has been no tradition of posting preprints or working papers in the life sciences, as there is in other disciplines such as physics or economics (Chiarelli et al., 2019; Xie et al., 2021).

Objectives

Against this background, we aimed to investigate the impact of COVID-19 preprint practices on scholarly publishing behavior, especially in the life sciences. For this, we conducted two studies:

  1. (1)

    We surveyed scientists and their experiences with posting preprints during the COVID-19 pandemic. In addition, we sought to gain insights into whether this proliferation of preprints in the life sciences—after having a pandemic bringing an external shock to the scholarly publication system—has marked the beginning of a cultural shift regarding posting of preprints or whether it is a temporary phenomenon.

In particular, we were interested in whether authors intend to continue to post preprints after the pandemic and whether preprint publication behavior was comparable to that happening during previous health crises. In addition, we wanted to find out whether such “external shocks” were more effective drivers of open access than, for example, institutional mandates to post preprints. In addition, we investigated what motivated authors to post preprints during the COVID-19 pandemic and whether answers are different for researchers who have started to post preprints before the pandemic and those who started during the pandemic. Survey answers were also compared between groups of researchers based on their gender, their career stage and region of origin (global south vs. global north).

  1. (2)

    We explored how the number of preprints has developed at different repositories and whether there were differences between COVID-19 and non-COVID-19 preprints, for example in terms of the number of authors per preprint or the number of citations the preprint received.

Related work

Posting of preprints gained momentum in the life sciences with the onset of the COVID-19 pandemic, with preprint servers hosting more than 30,000 COVID-19 related manuscripts within 10 months of the first confirmed COVID-19 case (Fraser et al., 2021). Unlike in economics or physics, where a strong preprint culture is prevalent, publishing of preprints was a rather new phenomenon for the life sciences (Puebla et al., 2022; Xie et al., 2021).

The obvious advantages of preprints are that they allow authors to disseminate their research results more openly and much more rapidly than journal articles, which usually require a lengthy peer review and publishing process (Puebla et al., 2022). However, there are also concerns that preprints are of lower quality than journal articles exactly because of the lack of peer review (Carneiro et al., 2020; Kodvanj et al., 2022; Nabavi Nouri et al., 2021). Furthermore, an extensive study on preprints from medRxiv and bioRxiv has shown that the impact of preprints on views, downloads, online mentions, and citations is strongly biased against preprints with authors from Chinese institutions (Fry & MacGarvie, 2023).

A number of studies have addressed the development of preprint servers in the life sciences (Fraser et al., 2021; Puebla et al., 2022; Sever et al., 2019; Xie et al., 2021). The pros and cons of preprints have been discussed (Kodvanj et al., 2022; Ni & Waltman, 2023), and studies have explored the motivations and concerns of authors with regard to posting or not posting preprints (ASAPbio, 2020; Chiarelli et al., 2019; Fraser et al., 2022; Rzayeva et al., 2023; Sever et al., 2019). Although the results of the numerous studies differ in detail and focus, the main benefit of preprints is considered to be the early and rapid dissemination of research results. Additionally, preprints support gaining early attention (Barrett, 2018; Chiarelli et al., 2019; Chung, 2020), claiming a priority (Vale & Hyman, 2016), receiving rapid feedback (Malički et al., 2021), and accumulation of citations due to their higher visibility (Fraser et al., 2022; Fu & Hughey, 2019). The lack of quality assurance and the potential risk of the media reporting incorrect results seem to be the most prominent aspects of preprint critique. Scooping risks and reliability as well as credibility issues are considered further disadvantages (Fraser et al., 2022; Puebla et al., 2022; Sever et al., 2019).

The findings of Fraser et al. (2021) “… show that preprints have been widely adopted for the dissemination and communication of COVID-19 research, and in turn, the pandemic has greatly impacted the preprint and science publishing landscape.” In addition, preprints contribute to the research community by providing a platform to share valuable research in a timely manner, which is extremely valuable during public health emergencies such as the COVID-19 outbreak (Xie et al., 2021). These two statements may sum up the current state of the debate on preprints in the life sciences.

Our study focused on the question of whether the increase in preprints in the life sciences is a one-time, quasi-crisis phenomenon or the beginning of a permanent (cultural) change in the publication behavior of researchers. To our knowledge, others have not addressed this question yet.

Methods and data

Survey with researchers who posted preprints on COVID-19 during the pandemic

The quantitative survey of authors of COVID-19 related preprints was conducted in September 2022. The 25 survey questions were grouped into five parts: (a) experiences with posting preprints during the COVID-19 pandemic, (b) motivations for posting preprints, (c) concerns about posting preprints, (d) role of research institutions or research funders, and (e) future development. The survey contained single and multiple choice questions, partially supplemented by a free text-option (see Data Availability Statement).

To automatically extract preprint data from open data sources (e.g., Crossref and DataCite) we used an already developed preprint tracker, which uses keyword queries to detect fitting preprints (Fraser & Kramer, 2020). We retrieved COVID-19 related preprints from bioRxiv, medRxiv, and Research Square. For bioRxiv and medRxiv, we extracted 24,436 COVID-19 related preprints for which we obtained 51,335 email addresses by scraping from repositories’ websites. After cleaning, 22,219 email addresses from bioRxiv and medRxiv remained. Two thousand additional email addresses from among a total of 11,194 randomly selected COVID-19 related preprints on Research Square were manually collected, since no automated solution was available. Ultimately, we sent the survey to 24,219 authors of COVID-19 preprints and received 1,131 completed responses (response rate: 4.9%), which formed the basis for our analysis. Free text answers were analyzed via an inductive coding process with two researchers. A codebook was developed based on Fraser et al. (2022) and applied to all free text answers (see Table 1, 2, and 3).

Table 1 Classification of free-text responses, in response to the question “What were your motives for posting a paper on a preprint server / repository?”
Full size table
Table 2 Classification of free-text responses, in response to the question “What concerns did you have about posting a paper on a preprint server / repository?”
Full size table
Table 3 Answers to the question “Do you intend to post your scientific work on preprint servers / repositories in the future?”
Full size table

To reveal differences between certain groups of respondents we compared answers from researchers who have posted their first preprint before the COVID-19 pandemic (N = 349) and answers from researchers who have posted their first preprint during the pandemic (N = 782) as descriptive analysis. Statistical comparisons were conducted for the following groups that usually expose distinct characteristics in publishing behavior: gender (female N = 329 vs. male N = 765; e.g., Armond & Kakuk, 2023), career stage (early career = phd, postdoc N = 272, later career = professor N = 609; e.g., Zhang & Glänzel, 2012) and region of origin (countries categorized as “global south” N = 265 or “global north” N = 831 according to UNCTADFootnote 2; e.g., Biesenbender et al., 2024). The grouping of answers is based on the self-selected answers of the survey participants. The latter three group comparisons were conducted with chi-square tests, which yielded significant results for p-values below 0.05% (see Appendix, Table 4 for the results). Answers that (strongly) agreed were summarized in “agree” and answers that (strongly) disagreed were summarized in “disagree”.

The majority of respondents came from the United States (25%), followed by the United Kingdom (10%) and Germany (7%) (Fig. 1). India, Canada, Spain, Italy, and France were next in line. Respondents were asked to use the Frascati Classification on Science and Technology (OECD, 2002) to assign themselves to a discipline: 666 respondents stem from the medical sciences, followed by 202 from the natural sciences, 101 from the social sciences and 81 from engineering and technology. About 70% of the respondents were male and 30% were female. 776 respondents were from universities, 240 from non-university research institutions. The response to the question “How many years have you been doing research?” showed that there are many respondents with quite long research experience. Four hundred and seven respondents reported doing research for 5 to 14 years (36%), 306 for 15 to 24 years (27%), and 298 for more than 24 years (26%).

Fig. 1
figure 1

Demographics of respondents: country of origin (a), discipline (b), gender (c), and institution (d)

Full size image

Quantitative analysis of preprint repositories

Our approach involved retrieving all preprints spanning the three years of the pandemic, from 2020 to 2022, across eight repositories with a significant collection of COVID-19 preprints: arXiv, Authorea, bioRxiv, JMIR Preprints, medRxiv, Preprints.org, Research Square, and SSRN. The repositories vary in terms of size, topical foci, and ownershipFootnote 3: bioRxiv, medRxiv, and JMIR focus on the life sciences (biology and medicine), whereas arXiv, Authorea, SSRN, Preprints.org, and Research Square cover multidisciplinary topics. Three of those repositories are led by scholarly institutions (arXiv: Cornell University, bioRxiv: Cold Spring Harbor Laboratory, medRxiv: Cold Spring Harbor Laboratory), while the other five are provided by major commercial publishers or private companies (Authorea: Wiley, JMIR preprints: JMIR Publications Inc, Preprints.org: MDPI, Research Square: private publishing services organization, SSRN: Elsevier). We accessed preprint data through the Dimensions database using the API and citation data was collected in 2023.

In total, we collected 1,195,274 preprints from these sources. After excluding 2,930 preprints with missing values (such as doi, author count, or citation frequency), our dataset comprised 1,192,344 preprints. To differentiate between COVID-19 and non-COVID-19 preprints, we employed the following search string in the preprints’ titles and, when available, in the abstracts: “coronavirus|covid-19|sars-cov|ncov-2019|2019-ncov|hcov-19|sars-2”. The subsequent analysis was conducted using R.

Results

In the following we will first summarize the results from the quantitative survey which will also highlight findings from the group comparisons and the survey’s free text answers. Second, we will report on the analysis of preprint servers and the characteristics the preprints expose.

Survey

Experience with posting preprints during the COVID-19 pandemic

The first part of the questionnaire was concerned with finding out when researchers started to post preprints and how frequently. The vast majority of respondents (69%) reported having posted a preprint for the first time during the pandemic (i.e., in the years 2020, 2021 and 2022; Fig. 2). In this group, the proportion of respondents from the medical sciences is particularly high at 64.3%, followed by the natural sciences (12.2%) and the social sciences (9.9%). In comparison: Only 46.8% of respondents who posted their first preprint before the pandemic came from the medical sciences, while the proportion of already preprint-experienced researchers from the natural sciences was comparatively higher at 30.9%, followed by 8.4% from the engineering sciences.

Fig. 2
figure 2

First time of posting a paper on a preprint server/repository (as of September 2022)

Full size image

When asked about how many preprints they have posted since the pandemic 81% of respondents said that they had published one to five preprints since the outbreak of the pandemic, and about 6% even said that they had published 10 or more preprints. In one of the survey questions, we asked for comparison with previous pandemics. Only 76 respondents answered, while 1055 respondents skipped the question. 1.8% of respondents have posted preprints on the ZIKA virus in 2016, 1.3% on Ebola virus in 2015, and 0.7% on swine flu (Johansson et al., 2018). This might be an indication that preprints have not played any role in previous health crises (maybe due to the lack of available repositories at that time); a finding which was also presented by Puebla et al. (2022) and Wang and Tian (2021, p. 8), who found that “fewer than 5% of articles were submitted to preprint platforms”.

In addition, we wanted to learn more about the researchers’ experiences with posting preprints during the COVID-19 pandemic. Thus, we were particularly interested in the differences between researchers who posted their first preprint during the pandemic and those who did so before the pandemic. In the following, we will analyze the survey responses for those distinct groups. Figure 3 shows the differences regarding the experiences with posting preprints in terms of visibility and citations received from the scientific community, attention from the broader public and media, and social media. About 57% of respondents, who already posted preprints before the pandemic, (strongly) agreed that their preprints were particularly visible to the scientific community during the pandemic. Respondents who posted their first preprint during the pandemic were even more likely to say so, with 69% agreeing. Significantly fewer respondents in both groups (strongly) agreed that their preprints were cited more frequently during the pandemic. Almost half of respondents (50%, respectively 46%) (strongly) agreed that their preprints were shared more frequently via social media, and that their preprints received a lot of attention from a broad audience and/or the media (44%, respectively 49%).

Fig. 3
figure 3

Experience with posting preprints during the COVID-19 pandemic in terms of visibility in the scientific community, social media, and the broader public/media. Results are shown for respondents who had posted their first preprints before the pandemic (a) and during the pandemic (b)

Full size image

With regard to the visibility of preprints during the pandemic the group comparisons showed the following significant differences (see Appendix Table 4 and Fig. 11):

  • Male researchers and researchers from the global south were more likely to agree that posting preprints during the pandemic made their preprints particularly visible to the scientific community and that preprints during the pandemic led to a lot of attention from a broad public and/or media—however, researchers from the global north were more likely to disagree with the latter aspect;

  • Later career researchers and researchers from the global south were more decisive regarding the impact social media had on the visibility of preprints: later career researchers and global south researchers were more likely to agree that their preprints were distributed more frequently via social media during the pandemic—at the same time, both groups were more likely to disagree with this statement than their counterparts (early career researchers and global north researchers respectively) which reveals a less neutral stance towards the impact of social media of those two groups.

Furthermore, we were interested in other potential benefits that researchers experienced from posting preprints during the pandemic, such as availability, public discussions, collaboration, journal invitations, and quality of preprints (Fig. 4). Most researchers agreed that posting preprints during the pandemic was essential to make their findings directly available to other researchers (86%, respectively 84% agreed/strongly agreed) as well as to society, which 65%, respectively 69% of respondents (strongly) agreed to be essential. Nearly 50% in both groups (strongly) agreed that a preprint served as a basis for collaborations with other researchers or led to invitations by journal editors to publish their preprint in an academic journal. Diverging experiences were found for whether posting preprints during the pandemic helped to ensure the quality of their publication: 27% of respondents who have already posted a preprint before the pandemic (strongly) agreed and 26% (strongly) disagreed, respectively. Slightly more respondents (32%) who posted their first preprint during the pandemic strongly (agreed) and 31% (strongly) disagreed. At 47% compared to 37%, respondents with more experience in posting preprints were more neutral in their assessment of quality.

Fig. 4
figure 4

Experience with posting preprints during the COVID-19 pandemic in terms of availability, public discussions, collaboration, journal invitations, and quality. Results are shown for respondents who had posted their first preprints before the pandemic (a) and during the pandemic (b)

Full size image

The group comparisons showed the following significant differences for experience with posting preprints during the COVID-19 pandemic in terms of availability, public discussions, collaboration, journal invitations, and quality (see Appendix Table 4 and Fig. 12a and b):

  • Early career research were more likely to agree that posting preprints during the pandemic was essential to make their findings directly available to other researchers;

  • Early career researchers were about 10% more likely than later career researchers to agree that posting preprints during the pandemic was essential for their research to be publicly discussed in society;

  • Global south researchers were more likely to agree with all the statements regarding experience with posting preprints during the COVID-19 pandemic in terms of availability, public discussions, collaboration, journal invitations, and quality than researchers from the global north.

Motivations and concerns regarding posting a preprint

By far the most important motivation for researchers to post a preprint was to increase awareness of their research (78%, respectively 75%) (Fig. 5), followed by “to stake a claim on my findings”. Respondents with longer posting experience were more likely to give this answer (60%) than those who were first posting preprints during the pandemic (48%). Responses to “receive early feedback” were fairly balanced. Publishing a more detailed version of a paper was not a strong motivation in both groups. To encourage citations or promote research collaborations were also not strong motivations to post preprints—especially not for respondents who posted their first preprints during the pandemic.

Fig. 5
figure 5

Motivations for posting a preprint. Results are shown for respondents who had posted their first preprints before the pandemic (a) and during the pandemic (b)

Full size image

When comparing the motivations for posting preprints across the three further groups of researchers a picture of distinct reasons appeared (significant differences between groups are presented, see Appendix Table 4 and Fig. 13):

  • Male researchers are more likely to indicate that their motivation to post preprints during the pandemic was to encourage possible research collaborations;

  • Early career researchers are more likely to indicate that they post preprints to stake a claim on their research findings and to publish a more detailed version of their research results;

  • Global north researchers are more likely to post preprints during the pandemic to increase awareness for their research and to stake a claim on their findings—whereas global south researchers are more likely to indicate that their goal was to receive early feedback, to publish a more detailed version of their research results and to encourage increased citations and research collaborations.

We received many optional free text responses to our question about the motivation for publishing preprints which we categorized (Table 1). There were many responses that emphasized speeding up the publication process [speed] and making research results immediately available [open science] as main motivations—both in terms of “research informing policy in a timely way” and “share our findings with [the] research community as early as possible”.

Respondents with longer preprint posting experiences were less likely to state that preprints lack quality assurance (35% in comparison to 49% for respondents who posted the first preprint during the pandemic) (Fig. 6). However, researchers from the global north were significantly more likely to worry about the quality of preprints than global south researchers (see chi-square test in Appendix Table 4) The risk of the media reporting incorrect results was also mentioned less frequently by respondents with more experience (39% compared to 44%), but early career researchers were more likely to agree with this concern (see Appendix Table 4 and Fig. 14). Only about 10% of respondents for both groups answered that comments and feedback on preprints are generally not helpful: later career researchers and researchers from the global south were more likely to agree with this statement (see Appendix Table 4 and Fig. 14). In addition, only 9% of respondents, who posted their first preprint before the pandemic, stated that: “A manuscript should only be published in a scientific journal if its content has not yet been posted as a preprint”. The chi-square test showed that global south researchers were more likely to state this. Conversely, this means that 92% of the respondents (implicitly) answered that the “Ingelfinger Rule” (which states that a scientific paper submitted for publication should not have been previously published elsewhere; Relman, 1981) no longer applies to them. The majority of respondents who posted the first preprint during the pandemic (83%) did not consider the Ingelfinger Rule important anymore or was not aware about it, but the share of respondents among this group who agreed with the statement was 8 percentage points higher than compared with the group of respondents who have posted preprints before. Biesenbender et al. (2024) could also show, that the Ingelfinger Rule still leads to a lot of confusion among researchers with regard to whether preprints are ‘harmful’ for later publishing.

Fig. 6
figure 6

Concerns regarding posting a preprint. Results are shown for respondents who had posted their first preprints before the pandemic (a) and during the pandemic (b)

Full size image

In the free text responses to our question about concerns regarding the posting of preprints, further worries were expressed and that we have not listed in our questionnaire, e.g., “Others would replicate findings and publish earlier” [competition] or “The paper could undergo revisions before actual publication. So the preprint and the final version will be different” [peer review process] (Table 2). There were a lot of worries about the journal policy, especially with regard to the Ingelfinger Rule, e.g., “Some journals react negatively to submissions posted in a preprint database” or “In certain journals the print was considered as duplicated version of the accepted manuscript”. The quantity of mentions of this concern is interesting, given that the majority of respondents did not agree with the survey question “A manuscript should only be published in a scientific journal if its content has not yet been posted as a preprint”. This hints towards a respondents’ lack of knowledge about current publishing practices and policies of journals.

In addition, we were interested in whether there is any quality control of preprints’ content and where this takes place. About 57% of all respondents answered that there is always some kind of quality control of content before their preprints are posted on a preprint server or repository. If we add the 13% of respondents who answered that quality control of content sometimes occurs, we can say that about 70% of preprints go through at least some kind of quality control. 56% reported that the quality control takes place in their research group, 43% answered that it happens at the preprint server.

Future development and sustainability

The survey should also reveal whether the increase in posting preprints is sustainable and how it will develop in the future. To investigate this, we asked researchers if they intend to post their scholarly work on preprint servers/repositories in the future (Table 3). 93.6% of respondents, who posted their first preprint before the pandemic, indicated that they will post all (41%) or some (52.6) of their future work on preprint servers. Respondents who had their first experience of posting preprints during the pandemic also largely agreed, but not to the same extent. Here, 73.3% of respondents stated that they would post all (15.3%) or some (58.4%) of their future work. In particular, they were less likely to answer that they would post all of their future work on preprint servers and they were more likely to say “no” or be unsure when compared with the group of respondents with more preprint experience.

In addition, we asked about the drivers that researchers believe are causing the increased posting of preprints in their field (Fig. 7). 70% of respondents with longer preprint experience (strongly) agreed with the statement that external shocks are powerful drivers, whereas a greater share of respondents (83%) who had posted the first preprint during the pandemic agreed with this statement. In comparison, only 42% of the latter (strongly) agreed that mandates or policies are effective drivers for changes in publication behavior. Among respondents with longer preprint experience, the difference between important drivers such as external shocks and mandates or policies was less noticeable: 58% stated that mandates or policies are driving preprint posting behavior. The situation was similar when considering disciplinary culture as an effective incentive for preprint posting: only 29% of respondents (with first preprint during the pandemic) compared to 47% (first preprint before the pandemic) (strongly) agreed that the disciplinary culture is a driving force for publishing preprints. When asked about what the researchers think about a cultural shift in publishing behavior in their field, 81% of respondents with longer experience (strongly) agreed that posting preprints will become common practice in their field. Less respondents who posted the first preprint during the pandemic agreed (61%). Slightly more than half of the respondents (64%, respectively 61%) (strongly) agreed that overlay journals can be a solution to combine the advantage of early access with improved quality control.

Fig. 7
figure 7

Sustainability of posting preprints in the future. Results are shown for respondents who had posted their first preprints before the pandemic (a) and during the pandemic (b)

Full size image

With regard to the sustainability of posting preprints the group comparisons showed the following significant differences (see Appendix Table 4 and Fig. 15):

  • Female researchers were more likely to agree that external shocks like the COVID-19 pandemic are powerful drivers for posting preprints in their field;

  • Men and researchers from the global north were more likely to disagree that mandates or policies are powerful drivers for posting preprints in their field—whereas female researchers were more likely to agree with this statement;

  • Global north researchers are more likely to agree that in five years time posting preprints will be a common practice in their research field;

  • Early career researchers and researchers from the global south were more likely to agree that overlay journals are a solution to combine the advantage of early access to preprints with improved quality control.

When taking into account all answers, surprisingly, our survey showed that mandates and policies play a minor role in incentivizing the publication of a preprint. To examine the role of research institutions and funders during the pandemic, we asked whether it is necessary for researchers to comply with an institution’s (Fig. 8, left) or funder’s (Fig. 8, right) open access/preprint policy: 77% of respondents indicated that it is not necessary; 81% pointed out that this is also true for funding agencies. Very few researchers reported that research funders or their own institution facilitate preprint publication.

Fig. 8
figure 8

Role of mandates and policies in preprint posting behavior. Results are shown for institutional mandates (a) and funder policies (b)

Full size image

Analysis of preprint servers

Development of preprint servers

We examined the quantity of COVID-19 preprints across each repository, as illustrated in Fig. 9. The data reveals that the peak number of preprints for each repository occurred at the onset of the pandemic in 2020, followed by a decline in 2021 and 2022. Notably, medRxiv had the highest volume of COVID-19 preprints, followed by SSRN and Research Square. Conversely, the corresponding graph for non-COVID-19 preprints (Fig. 9, right), presents a more nuanced scenario. Among the most popular repositories in terms of size, namely arXiv, Research Square, and SSRN, there was a slight increase in the number of preprints—with SSRN having experienced a notable rise in 2022. No discernible change was evident for the other five repositories. Regarding non-COVID-19 preprints, arXiv had the highest volume, followed by SSRN and Research Square. bioRxiv exhibited some variance. An exception was observed in the case of medRxiv, where more COVID-19 preprints were published compared to non-COVID-19 preprints, likely due to the repository’s thematic focus.

Fig. 9
figure 9

Annual breakdown (2020–2022) of COVID-19 preprints (a) and non-COVID-19 preprints (b) for eight repositories. Figures differ in the scales of the y-axes for the number of preprints

Full size image

Characteristics of COVID-19 and non-COVID-19 preprints

Figure 10 on the left reveals no disparities between COVID-19 and non-COVID-19 preprints concerning the number of authors per preprint, with the exception of arXiv, where high author numbers were evident. This was primarily attributable to disciplines such as Particle and High Energy Physics which form a large user group of arXiv. Conversely, Fig. 10 on the right illustrates a significant distinction in citation numbers between COVID-19 and non-COVID-19 preprints (as of 2023). Across repositories, COVID-19 preprints received an average of 5.14 citations (min = 0/max = 1214/SD = 22.72), while non-COVID-19 preprints received an average of 0.31 citations (min = 0/max = 960/SD = 2.15). Notably, arXiv and Authorea exhibited low citation counts for both types of preprints, potentially reflecting the citation practices within the covered disciplines that may not formally cite preprints or because of the low amount of COVID-19 preprints posted on those two platforms overall. Interestingly, one of the most cited COVID-19 preprints is found on SSRN (https://doi.org/10.2139/ssrn.3557504 published in 2020 received 1214 citations), which discusses the economic impacts of the pandemic. At the same time, it is currently the most cited preprint on SSRN, reflecting the relevance of research on the pandemic also outside the life sciences.

Fig. 10
figure 10

Preprints’ author counts (a) and number of citations (b) in eight repositories published between 2020–2022

Full size image

Discussion and future work

We set out to study the impact of COVID-19 on preprint practices and on scholarly publishing behavior, especially in the life sciences, with a mixed-method approach. There were some early indications that the COVID-19 pandemic has triggered a significant increase in the number of preprints, which, while not remaining at the same level, seemed to be translating into a sustained change in publication behavior in the life sciences (Fraser et al., 2021). Our survey and the analysis of preprint repositories have, however, arrived at somewhat contradicting results—also, responses from researchers who have gained experience with posting preprints before the pandemic differed widely in some aspects from those respondents who started to post preprints during the pandemic. Additionally, researchers’ gender, career stage and region of origin influenced how they consider the role of preprints during the pandemic and after.

From our survey we can conclude that the vast majority of respondents (69%) who posted a preprint on bioRxiv, medRxiv, or Research Square did so for the first time during the pandemic. This is consistent with other studies that have shown an increase in preprints at the beginning of the pandemic (Fraser et al., 2021; Waltman et al., 2021). The survey results also clearly show that preprint posting behavior was not comparable to that from previous health crises where only few respondents deposited preprints to disseminate results quickly (Puebla et al., 2022; Wang & Tian, 2021). However, respondents reported good experiences with posting preprints during the COVID-19 pandemic, e.g., many respondents found that their preprints are particularly visible—an experience which was especially supported by respondents who posted their first preprint during the pandemic as well as male researchers and researchers from the global south. The question about the role of social media in increasing visibility of preprints triggered stark opinions of later career researcher and researchers of the global south who either completely agreed or disagreed with the statement. Survey respondents also pointed out that making their findings directly available to other researchers is a major motivation, especially for early career researchers, which has been shown in other studies as well (Fraser et al., 2022; Waltman et al., 2021). Overall, early career researchers and researchers from the global south were more likely to agree with statements concerning the experiences with posting preprints during the COVID-19 pandemic in terms of availability, public discussions, collaboration, journal invitations, and quality. Although our preprint server analysis has shown that COVID-19 preprints received far more citations, this has not been a strong motivation for respondents to engage in preprint publishing, even less so for respondents who posted their first preprints during the pandemic.

On the other hand, the survey revealed that an external shock, like the COVID-19 pandemic, is considered a more effective driver for open access and preprint publishing than mandates or funders’ policies—in fact, respondents predominately indicated that they do not have to comply with institutions’ or funders’ policies/mandates during the pandemic. Also, many respondents confirmed that publishing an article as a preprint does not stand in the way of publication in a journal (so-called Ingelfinger Rule). Interestingly, the group comparisons revealed some particularities in this regard: female researchers as well as global north and global south researchers were more likely to agree with the COVID-19 pandemic being a driver for preprint publication in their field. On the other hand, male researchers (as opposed to females) and global north researchers were rather strongly rejecting the influence of mandates and policies on current and future preprint publishing behavior.

In terms of future developments, it is also interesting to note that almost 50% of respondents (strongly) agreed that posting preprints during the pandemic has led to invitations from journal editors to publish their preprints in a scientific journal. This could be an indication of the increasing number of overlay journals (Rousi & Laakso, 2022). But—and we speculate here and hope for further in-depth research to shed light on this phenomenon—it could also hint towards (sometimes lower quality or predatory) journals that offer to publish preprints as articles without peer review and/or in exchange for an article processing charge.

We found differing opinions about whether posting preprints improves the quality of research and published research results: only about a quarter to a third of the respondents (preprint experience before pandemic vs. preprint experience during pandemic) agreed with this statement and about the same amount disagreed. So, as others have found, preprints raised concerns about quality control as well as publicly presenting scientific results that have not undergone peer review (Fraser et al., 2022; Penfold & Polka, 2020). Interestingly, respondents with longer preprint posting experience were less likely to state that preprints lack quality assurance—on the other hand, researchers from the global north were significantly more likely to worry about the quality of preprints than global south researchers (a similar result was found in Biesenbender et al., 2024). However, 70% of our respondents pointed out that preprints are subject to at least some kind of content quality control carried out by the scientific community (a bit more than 50% referred to an internal peer review process). This corresponds to the fact that other studies also found only little evidence that preprints are of lower quality than published articles: the works of Brierley et al. (2022), Xie et al. (2021), and Kodvanj et al. (2022) seem to indicate that the quality concerns expressed by researchers are more a gut feeling than an empirically validated finding. Considering that a lack of quality control was also one of the main reasons for concerns about preprint publishing our respondents mentioned in our study, we find this result quite remarkable. Further research should therefore investigate how those perceptions form within researchers and how quality control of preprints is carried out in practice. Additionally, we found some significant differences in answers to the role overlay journals may play in future with regard to combing the advantage of early access to research results with improved quality control. Early career researchers were more likely to see this advantage, which may hint towards a change in perception and needs with the research community—maybe triggered by the COVID-19 pandemic since overlay journals were found to be niche products in 2021 (Rousi & Laakso, 2022; Ursić et al., 2022).

The experiences with posting preprints before and after the COVID-19 pandemic may have encouraged the majority of respondents (79%) to indicate that they will post at least some of their future work as preprint, which was also found by others (e.g., Waltman et al., 2021). This survey result, however, is mainly affected by the answers of those respondents who gained preprinting experience before the pandemic (93.6% indicated that they will publish all [41%] or some [52.6%] of their work as preprints). Only 15.3% of preprint novices stated that they will post all or some (58.4%) or their work in future as preprints. A similar result appeared when asked about how preprint repositories will change the disciplinary publishing culture in future: while 81% of preprint experts (strongly) agreed that preprint publishing will become common practice in their field (which was also significantly supported by researchers from the global north) only 61% of preprint novices supported this statement. One reason for this could be the distribution of disciplines in the two groups. In the group of the preprint novices, the proportion of respondents from the medical sciences, in which the posting of preprints is a relatively new phenomenon, is 17.5 percentage points higher than in the group of respondents with longer experience in posting preprints, while the proportion of respondents from the natural sciences, in which the publication of preprints is a more established practice, is 18.7 percentage points lower in the group of the novices.

This nuanced picture of preprint publishing behavior and attitudes towards preprints, may explain the results we have presented in the second part of our study, the analysis of eight preprint servers. Our analysis showed that the substantial surge in the volume of COVID-19 preprints, particularly at the onset of the 2020 pandemic, has not resulted in a significant overall growth in preprint repositories, especially not within the life sciences where we found stagnating growth numbers.

Reasons for that may be rooted in different areas of publishing practice in the life sciences and which require further investigation. There may be editorial practices of life science journals in place that prevent researchers from publishing preprints, although our survey indicates that the Ingelfinger Rule does not affect authors a lot. Journals may have developed new publishing processes which result in more rapid publication of research results and that may reduce or replace the need for preprints (Miller & Tsai, 2020). In contrast to other disciplines, such as physics, mathematics, computer science, or economics, the life sciences may just have different needs in terms of publishing, dissemination and discussion of research results as well as reputation management which are not (yet) addressed (sufficiently) by the available preprint servers. Additionally, the preprint server landscape for the life sciences has undergone substantial diversification with bioRxiv founded in 2013 (Puebla et al., 2022), JMIR Preprints in 2009, and medRxiv in 2019 which may have distorted preprint publishing practices. Our results can, of course, also be reflective of our research methodology that has limited itself to eight preprint repositories and only three years of investigation. We may have neglected the right or enough preprint servers to arrive at meaningful results for the life sciences and other disciplines. Future research should include Zenodo, amongst others. Also, change may take time and three pandemic years might not be enough to reflect the evolution in publishing behavior that might actually take place in the life sciences right now.

From our quantitative results we may also conclude that the enthusiasm for preprints in the context of COVID-19 does not seem to have extended to other research areas, as the slow increase in the number of non-COVID-19 preprints reflects and which seems to follow the usual growth rates of those repositories in general (Xie et al., 2021). An exception was observed for SSRN, which experienced a considerable increase in preprints in 2022, the reasons for which may vary but may be a result of a backfilling effect (Archambault et al., 2014).

COVID-19 preprints and non-COVID-19 preprints do not differ substantially in terms of the number of co-authors (except for arXiv, which exposes large author counts due to disciplinary norms), which may indicate that preprints were posted for other reasons than increasing the amount of authors’ outputs. However, COVID-19 preprints received substantially more citations on average than non-COVID-19 preprints, most likely due to early accessibility and speed of publication processes during the pandemic.

Conclusion

With regard to the long-term development and sustainability of preprint publishing practices, our mixed methods-study has shown that publishing behavior, and along with it preprint publishing, seems to be more strongly influenced by disciplinary norms and practices (Ni & Waltman, 2023), as well as the respondents’ region of origin and career stage, than by external shocks as the COVID-19 pandemic. Although the COVID-19 pandemic was certainly a powerful driver for preprint publishing in the life sciences and although there is a majority of researchers who are willing to continue preprint publishing after the pandemic, both influences were not sufficient enough to change the disciplinary culture in the long run, especially not in the life science (at least it is not yet visible in the growth rates of preprint repositories). Also, open access/preprint mandates and policies by institutions and funders were not considered important for decision making in publishing, which is surprising given the effort science policy is investing in this regard. Since our study has only included three years of preprint publishing and a small set of repositories, large long-term studies are needed and essential to comprehensively understand how repositories will evolve in the future, addressing questions related to the sustainability of preprint posting across various disciplines (Xie et al., 2021). Furthermore, the effect of the COVID-19 pandemic has to be studied more comprehensively, for example with regard to the time researchers could spend on their research during lockdowns and the effect on different disciplines, genders, and researchers with care responsibilities (Myers et al., 2020). Since it has been shown that preprint authors from Chinese institutions receive far less recognition (Fry & MacGarvie, 2023) and since our survey only received 0.15% (165 respondents) of answers from Asian authors (Top 4 countries in our sample: 59 respondents from India, 12 from Israel, 8 from Japan, 8 from China) making it not indicative in this regard, it would also be interesting to learn more about the experiences of Chinese preprint authors (and those of other underrepresented countries) in future large-scale studies.