During the early years of the COVID-19 pandemic, major investments were made in research aimed at understanding the epidemiology of the disease and developing vaccines, anti-viral therapies, and diagnostic technologies. To achieve this, preclinical and clinical (experimental and observational) research processes were sped up and scaled up in both the public and private sectors and in partnerships between them (Slaoui and Hepburn 2020; Agrawal, et al. 2021). In addition to major research funding initiatives, processes were put in place to coordinate research efforts (National Institutes of Health 2020), to speed up publication (Horbach 2020; Johansson and Saderi 2020), and to share data through “open science platforms” (Capps 2021). Research was also orientated towards COVID-19 studies and away from other diseases and questions, and, in some settings towards study designs that would generate data on multiple interventions as quickly as possible (e.g. platform trials). Within just a few months of the pandemic’s onset, well over 1000 studies had been registered in the international clinical trial registry, ClinicalTrials.Gov (Bramstedt 2020). As a result of these efforts, a suite of diagnostics, vaccines, and antiviral treatments were available less than two years into the pandemic.

While there are no simple ways to assess the rigour, integrity, and ethics of an entire body of research, it is likely that most pandemic research was conducted rigorously, collaboratively and with due regard to standards of ethics and research integrity. Had this not been the case, it is unlikely that there would have been so many diagnostic and therapeutic successes and research-related scandals would have been far more prevalent. Indeed, the pandemic showed the potential for unprecedented levels of cooperation and goodwill (as evident, for example, in the agreement of most medical journals to publish COVID-related articles in open access format, sequence sharing, and some intellectual property agreements). At the same time, it is clear that the pandemic pushed some aspects of research methodology, ethics, and integrity to their limits and revealed areas of pre-existing uncertainty, inconsistency, and deficiency. While it is important not to discount the extraordinary successes of pandemic research, it is equally important to systematically examine what deficiencies in research were exposed or created by the pandemic, so that we can learn from these.

Pandemic Research Ethics

Not surprisingly, as research was sped up and scaled up to address the pandemic, ethical questions arose about whether to continue non COVID-19-related studies (Bierer, et al. 2020); how much funding to allocate to research (as opposed to clinical care and public health interventions) (Bierer, et al. 2020; Bunnik and Smids 2021); and how to treat research participants. With respect to the latter, there were questions about whether usual consent and privacy provisions could be waived for some kinds of research (Singh, Cadigan, and Moodley 2022; Tosoni, et al. 2022); how much risk was acceptable for research participants (Jamrozik, Heriot, and Selgelid 2020; Menikoff 2020); and what degree of scientific and clinical uncertainty were needed to justify conducting clinical trials or continuing them as data emerged and clinical and epidemiological circumstances changed (Lenzer 2020).

Social justice issues also loomed large in ethical discussions, with questions being raised about how to ensure equity of access to research (Spector‐Bagdady, et al. 2022; Wieten, Burgart, and Cho 2020), how to empower and protect participants who come from “vulnerable” and “marginalized” groups (Crooks, Donenberg, and Matthews 2021; Faust, et al. 2021; Pratt and Bull 2021; Singh, Cadigan, and Moodley 2022), when and how to involve communities in research agenda setting, design, and ethical assessment (Lee and Eyal 2021; Pratt and Bull 2021; Straiton, et al. 2020) and what, if any, access to interventions was owed to research participants (Zaidi, et al. 2021). There were also broader social justice concerns raised about nationalistic research and development agendas (Hafner, et al. 2020) and the focus of the global research effort on technological solutions that both failed to address the needs of the most vulnerable populations (e.g. adequate housing to allow for social distancing or management of comorbid conditions (Patel, et al. 2020)) and were not made available to all who can benefit from them (Mathieu, et al. 2021). The global harms of such oversights are only now becoming apparent (Mahase 2022).

Importantly, none of these research ethics issues were new or unique to the pandemic. For example, it has long been recognized that consent to research might be waived if risks are low and there is sufficient public interest (e.g. research using previously collected de-identified datasets). Similarly, issues to do with research funding, agendas, and priorities, acceptable risk and uncertainty, and distributive and procedural justice are core research ethics concerns. What might have been different during the pandemic was the scale of the issues given the number of people potentially affected and the speed with which ethics reviews needed to be conducted. It appears that, whether or not COVID-related trials raised any novel ethical issues, many human research ethics committees lacked both the expertise and the resources necessary to conduct rapid appraisals of COVID-19 trials (Faust, et al. 2021). It is also noteworthy that pre-existing national and international pandemic plans, policies, and processes—including those that pertain to research in public health emergencies (Nuffield Council on Bioethics 2020)—were not consistently used during the COVID-19 pandemic and “reinvention of the wheel” was common when it came to research ethics (and other) policies and processes.

Rigour and Integrity in Pandemic Research

Given that the clinical, scientific, and ethical stakes were so high, it was crucial that scientists could be trusted to design, conduct, and disseminate their work rigorously and with integrity. Sadly, however, the behaviour of a small number of researchers during the pandemic showed that scientific quality and integrity could not always be taken for granted. For example, early in the pandemic high-profile peer-reviewed articles examining the safety and efficacy of hydroxychloroquine were retracted by the Lancet (Mehra, et al. 2020c; Mehra, Ruschitzka, and Patel 2020a, b, c, d) and the New England Journal of Medicine (Mehra, et al. 2020b; Mehra, et al. 2020a) after other academics and journals raised questions about the integrity and validity of the datasets used, the statistical analyses conducted and the conclusions drawn (Watson, et al. 2020; The Lancet 2020; Davey 2020; Mehra, Ruschitzka, and Patel 2020a, b, c, d; Ledord and van Noorden 2020; Davey, Kirchgaessner, and Bossley 2020).

While overt cases of misconduct were, fortunately, rare, more subtle problems with the planning, coordination, design, review, conduct, and dissemination of research were far more of an issue (Herper and Riglin 2020; Bierer, et al. 2020; Glasziou, Sanders, and Hoffmann 2020). For example, early clinical trials into the use of azithromycin/hydroxychloroquine (Gautret, et al. 2020), remdesivir (Grein, et al. 2020), and convalescent plasma (Sullivan and Roback 2020) were all confounded by small sample sizes, a lack of randomization, missing data, and epidemiologically unsound inclusion and exclusion criteria (Dinis-Oliveira 2020; Angus, Gordon, and Bauchner 2021; Pundi, et al. 2020; Bierer, et al. 2020). There were also concerns raised about replacing or supplementing traditional randomized trials with single arm trials, observational studies, and real-time data analytics. Here, the concern was that standards of evidence were being unjustifiably compromised in the pursuit of rapid answers (London and Kimmelman 2020; Smith, Rakestraw, and Farroni 2022; Bramstedt 2020).

Importantly, it was not only interventions that were quickly debunked (such as hydroxychloroquine and ivermectin) that were arguably not investigated as thoroughly as they could have been (even in the context of a public health emergency). Evidence is now emerging that “mainstream” interventions, which were widely supported by the scientific community and taken up into practice around the world, drew upon at least some data that was of poor quality and biased. For example, some modelling studies to predict morbidity and mortality that were used as the basis for public policy responses, such as isolation, vaccination, and treatment of COVID-19, were biased and used poor controls, inappropriate inclusion and exclusion criteria, and poor statistical analysis (Wynants, et al. 2020); some early phase studies to determine vaccine doses were heterogeneous and underpowered (Dunn, et al. 2022); and studies of the antiviral molnupiravir over-estimated the benefit of the treatment, which was subsequently shown to be likely no better than placebo (Butler, et al. 2022; Wise 2022).

Weaknesses in scientific rigour and integrity were also evident in research authorship, peer review, and dissemination. In some cases, there was slow research publication and insufficient data sharing (Sumner, et al. 2020) due to, for example, the need in some countries for authors to obtain government permission prior to publication and limitations in some jurisdictions on scientists’ freedom to contribute to public policy debates. At the same time, there was also evidence of a rush to publish, with authors of some COVID-related studies failing to adequately review data prior to publication (Mehra, et al. 2020a; Mehra, Ruschitzka, and Patel 2020a, b, c, d) and journal peer review processes being compromised by efforts to speed them up or bypass them altogether with publication prior to review (sometimes in poor quality pre-prints that were not subsequently formally published) (Abritis, Marcus, and Oransky 2021; Agoramoorthy, Hsu, and Shieh 2020; Bramstedt 2020; Dinis-Oliveira 2020; Yeo-Teh and Tang 2021; El-Menyar et al. 2021). Other problems with information dissemination were poor authorship practices such as “guest” authorship (Papadakis 2021) and misrepresentation of research findings (for example, “spinning” conclusions in medical journals (Bero, et al. 2021; Dinis-Oliveira 2020) and in public fora (Grady 2021; Caulfield, et al. 2021; Parker, et al. 2021)).

These overt and more subtle failures of scientific rigour and integrity had many undesirable effects: Most obviously, research that was poorly designed or otherwise lacking in integrity slowed down the search for effective interventions, wasted resources, and exposed research participants to unjustifiable risks (Herper and Riglin 2020; Bierer, et al. 2020; Glasziou, Sanders, and Hoffmann 2020; Di Girolamo and Meursinge Reynders 2020; Pundi, et al. 2020). Failures of scientific rigour and integrity also likely distorted processes that rely on scientific information such as health technology regulation, resource allocation, clinical guideline production, and research agenda setting (Mahase 2020; Singh and Ravinetto 2020; Herper and Riglin 2020).

When the public was alerted to failures of rigour or integrity—for example, when high profile papers were retracted—this likely contributed to erosion of public trust among some groups (which was already fragile due to misunderstandings and general mistrust of authority). In this regard, it is important to recognize that it is difficult to argue for fallibilism and for public acceptance that many results in science will ultimately be overturned, in a context where there is outright fabrication or even insufficient rigour. Public awareness of lack of rigour and integrity may also have fomented conspiracy theories, made it more difficult to recruit participants into subsequent studies, and threatened compliance with public health advice (Seale, et al. 2020; Goldenberg 2021; Hornsey, Lobera, and Díaz-Catalán 2020; Caulfield, et al. 2021; Pickles, et al. 2021). Irresponsible scientific communication may also have contributed to already extensive public misinformation (Pickles, et al. 2021; Singh and Ravinetto 2020), created political and public demand for ineffective interventions (Mendel, et al. 2021; Mahase 2020) and reduced people’s motivation to participate in research because they believed that there were effective interventions available (Ledford 2020). It may also have impacted negatively on trust because conflicting messages were being disseminated with such certainty and the uncertainty that is inherent in the scientific process was poorly communicated (London 2021; Veit, Brown, and Earp 2021).

The most obvious explanation for these problems is simply that there was more science being conducted more quickly and with less peer and institutional oversight. Importantly, there is a mutually reinforcing and compounding relationship between science and subsequent public health decision-making, regulation, and subsidization of interventions. If public health officials, regulators, and payers are calling for the rapid generation of data and are willing to act on the basis of low standards of evidence (often under pressure from industry and consumer activists), then there is an incentive for scientists and publishers to cut corners. This system, in which science, commerce, and policy act mutually to erode evidence standards was clearly evident during the COVID-19 pandemic.

Were These Problems Unprecedented, Or Was it Business as Usual?

As we and others have argued, many of the failures of scientific rigour and integrity that occurred during the pandemic were exacerbated by the rush to generate, disseminate, and implement research findings, which not only created opportunities for unscrupulous actors but also compromised the methodological, peer review, and advisory processes that would usually identify sub-standard research and prevent compromised clinical or policy-level decision-making (Watson, et al. 2020; Agoramoorthy, Hsu, and Shieh 2020; Abritis, Marcus and Oransky 2021; Bramstedt 2020; Dinis-Oliveira 2020; Caulfield, et al. 2021; Lipworth, et al. 2020).

While it would be tempting to attribute these failures of science and its translation solely to the “unprecedented” circumstances of the COVID-19 pandemic, the reality is that there have been moves for decades to speed up research, to limit or “streamline” ethics review, to accelerate the dissemination and translation of research into policy and practice (Morris, Wooding, and Grant 2011; Kessler and Glasgow 2011; Kesselheim and Avorn 2017), and to encourage new kinds of collaborations between academic disciplines, and between academic, healthcare, commercial, consumer, and political organizations (Minkler 2005; Johnson, et al. 2014; Oliver, Kothari, and Mays 2019; Churruca, et al. 2019; Dove and Özdemir 2015).

It was known before the pandemic that training in research methods and research integrity are not always integrated into research training programmes, compliance with institutional research integrity policies can be haphazard, and policies and processes often tend to emphasize governance and risk management more than ethics (Roje, et al. 2022; Anderson, et al. 2013).

The failures of scientific rigour and integrity that have occurred during the pandemic are, therefore, not new, and will not disappear when this pandemic has run its course. This raises the question: how can scientific rigour and integrity be understood, motivated, structured, supported and, where necessary enforced in research that is accelerated and scaled up?

Rethinking Integrity in Sped-Up and Scaled-Up Research

To answer this question, it is helpful to begin with lists of principles that have been generated to encourage scientific rigour and guide research integrity. One such list is that of the InterAcademy Partnership (InterAcademy Partnership (IAP) 2016), a global network of more than 140 scientific academies, which agreed upon a set of seven principles to guide global research:

  • honesty—conducting research and communicating without deception,

  • fairness—treating others with respect and without bias

  • objectivity—trying to look beyond one’s own conceptions and biases

  • reliability—adhering to methods that produce trustworthy results

  • scepticism—continually re-examining and improving results and explanations

  • accountability—being willing to justify results and conclusions to other researchers and to society more generally

  • openness—making data and other information underlying results publicly available.

While the importance of these principles in any type of research appears undeniable, they need to be specified in terms of precisely what they mean in the context of sped-up and scaled-up research and publication, what strategies are likely to work for their enactment and enforcement, and who should be responsible for their oversight. Existing strategies for promoting scientific rigour and integrity (such as education, mentoring, organizational culture change, arms-length funding agreements, peer review, study registration, data sharing, funding disclosures, reporting standards and auditing retraction, community engagement, whistle-blower protections and sanctions for wrongdoing (Kretser, et al. 2019)) might need to be more rigorous, better integrated into research training and institutional cultures, and more sophisticated—for example, recognizing the degree to which ethics, epistemology, and policy are related.

Principles used to promote rigour and integrity might also need to be modified or supplemented with other principles and strategies that are fit for purpose in research that is funded, conducted, reviewed, and disseminated by stakeholders with a wide range of interests and agendas. For example, while some sense of objectivity remains an important goal in any kind of research, it is important to be realistic about what objectivity actually means and what degree of objectivity is actually possible when commercial actors, consumer advocacy groups, and other political actors not only support research (through financial and in-kind contributions) but also “co-design” and “co-produce” it and assist with “knowledge transfer” (Douglas 2009; Daston and Galison 2021; McCabe, Parker, and Cox 2016). Indeed, one could argue that the goal of “objectivity” should be replaced with the goal of “managed influence and bias” to alert people to the realities of the environment in which they are operating. Situations such as the COVID-19 pandemic might also prompt us to return to foundational questions about subjectivity and “values in science” (Douglas 2009; Daston and Galison 2021) and determine which of these need to be accepted, which need to be managed, and which might even be harnessed for their potential benefits. Perhaps the role of researchers, as part of a research community, should be “to improve the social mechanisms that allow for the exposure and critical evaluation of the values that necessarily persist in medical research” (Borgerson 2017, 319), thereby taking seriously the role of values and the fallibility of science. To the extent that objectivity remains a goal, it will be important to strengthen existing strategies for addressing biases and influence and ensuring that they are enforced. For example, processes such as peer review, study registration, data sharing, funding disclosures, reporting standards, auditing, and retraction will need more systematic attention and resourcing in this context.

Likewise, the notion of, and commitment, to “openness” in research might need some re-thinking when research is sped-up and scaled-up. While this principle is relatively unproblematic when it is conceptualized in terms of researchers sharing data with one another to facilitate peer review, disseminate innovative ideas, and prevent undue commercial or political secrecy, it can also be problematic when only some groups of scientists have the resources to practice openness (at least in regard to publication) and when open access to data can be taken advantage of by those who have commercial and political interests and who use data without simultaneously contributing to the public good. For example, while there is nothing wrong with a company benefiting from a publicly shared data source to develop a product, it is more problematic if they use this publicly accessible information to develop products that are priced beyond the means of individuals, health systems, or even nations. It is also problematic if they use the data for purposes that have no public interest (e.g. racial profiling) or fail to govern data exchanges in responsible and mutually beneficial ways. In this way, notions such as an “open commons” can be co-opted by commercial or political entities and it is arguable that such entities should be “excluded from the commons and its benefits” (Capps 2021, 11).

Existing ideas about openness may also require moderation when it comes to communication of research results with the public. During the COVID-19 pandemic, scientific experts faced the dilemma of, on the one hand needing to encourage public action (e.g. to socially distance, wear masks, and vaccinate) by conveying scientific information as it emerged. On the other hand, they needed to avoid changing their positions too frequently and may, at times, have been justified in temporarily withholding information in order to ensure that their messages were consistent, coherent, and perceived by the public to be valid (Desmond 2021). This challenge is not unique to the pandemic, and there will always be a need to balance openness with prudence when it comes to public communication.

At the same time, it will also be important to consider the roles and responsibilities of academic and government communications offices and the media (for example, whether they need to be more regulated when it comes to the reporting of health information) and enhance public understanding of science so that the public is better able to understand how ideas emerge and evolve, and that changes in information and understanding may be signs of scientific integrity, rather than signs of its absence (Kitcher 2001) (thereby countering President Donald Trump’s assertion that, “listening to scientists” is something that only a fool would do (Webb and Kurtz 2022) ). This is, of course, likely to be an enormous challenge given the pervasiveness of social media and the varying scientific literacy and views about science in the community. In this regard, as with other areas of science and health, some people have a good understanding of science, others have a very limited understanding of science, others (including recent world leaders) have a profound distrust of science or a desire to undermine it, while others have a naive trust in science and the scientific method (Kandel 2021; Nature Medicine (editorial) 2001).

In addition to reconfiguring existing principles and processes of research integrity for sped-up and scaled up science, it might also be necessary to introduce new ones. For example, the virtue of “intellectual humility” might need to be added to lists of integrity-related principles because, as science speeds up and scales up, it will be increasingly difficult for collaborators, peer reviewers, and users of research, to fully understand the work they are facilitating, reviewing, or translating into practice.

More generally, principles of research integrity might need to more explicitly acknowledge the degree to which science is an ecosystem that relies on preparation and coordination and the cooperation of communities of scholars who are needed not only to operationalize research integrity (e.g. by participating in peer review processes) but also to define it and advocate for it. For example, it might be possible, for the scientific community, along with national and global policy organizations, to make more systematic use of clinical trials with open, adaptive platforms that allow multiple drug or vaccine efficacy assessments or diagnostic testing assessments that can be designed ready for rollout. There is evidence from the COVID-19 pandemic that such approaches are feasible and fruitful—as evident, for example, in the Solidarity trial (WHO Solidarity Trial Consortium 2022) and the U.K. Recovery trial (University of Oxford). It might also be possible to better prepare clinical and research databases (and put in place the necessary ethical oversight mechanisms) so that, during the next health emergency, human and pathogen genomic databases can be more efficiently integrated with clinical, public health, and administrative databases. While this kind of preparation and coordination requires resources and independence of these processes from those with commercial and political agendas, there was evidence during the COVID-19 pandemic that researchers’ skills (for example in genomic sequencing) and research resources established for other purposes could be harnessed to facilitate COVID-related research.

Importantly, operationalization of any reworked research integrity principles will need to take place at multiple levels—including in the creation of broadly shared codes of ethics and at more local levels. Both of these approaches have strengths and limitations—for example, broadly shared codes of ethics have gravitas but are not always sufficiently nuanced for local contexts and may not be perceived as legitimate by all stakeholders. Locally developed policies and processes are more likely to be accepted and actionable but can be idiosyncratic and exceptionalist and may not reflect broadly shared values and norms.

In suggesting that principles of research integrity might need to be problematized and reconceptualized—not just in pandemics but in science more generally—it is crucial not to lose sight of their value. But equally, simply asserting principles of scientific integrity without due regard to context and change renders them hollow and might itself leave science vulnerable to manipulation or to being dismissed.

The COVID-19 pandemic has made clear both the stability of science and its capacity to be mobilized, and the ever-shifting social determinants of its standards, norms, and virtues. Ideas about research integrity that were developed when scientists worked at their own pace and in relative isolation may no longer be fit-for-purpose. By critically examining principles of research integrity, it should be feasible to continue to speed up and scale up science and its translation without sacrificing rigour or integrity. At the same time, in suggesting that research integrity might need to be bolstered, not just in pandemics but in science more generally, it is crucial not to lose sight of the enormous contributions that science has made. Doing so would only play into the hands of those who wish to undermine science, as was the case with anti-science movement and right-wing media and politics during the COVID-19 pandemic.