Expert Forecasts of COVID-19 Vaccine Development Timelines

INTRODUCTION

As of July 2020, the COVID-19 pandemic has resulted in close to 12 million infections and more than 500,000 deaths globally.1 Rapid development of a SARS-CoV-2 vaccine is a public health priority.

Because of the unprecedented nature of research efforts, historic experience with vaccine development may not provide accurate estimates for SARS-CoV-2 vaccine development timelines and outcomes. Expert elicitation and aggregation synthesize diverse insights that are spread across expert communities, and offer an alternative way to estimate when vaccines will be available.2,3

METHODS

We identified experts through the contact network of the vaccinologist author BJW and corresponding author searches for review articles on vaccinology in highly cited medical journals. We solicited experts for participation by e-mail on the week of June 21st and a second time on the week of June 28th.

We asked for timeline forecasts for three vaccine development milestones: (1) a field study enrolling at least 5000 participants reporting results, (2) a vaccine being available to those at highest risk from the virus in the USA and/or Canada, and (3) a vaccine being available to the general public in the USA and/or Canada. Experts provided best, soonest, and latest estimates for when each milestone would occur. We also asked for probability estimates of two setbacks: (1) the first vaccine widely deployed in the USA and/or Canada receiving a boxed warning from the FDA to highlight serious or life-threatening adverse reactions and (2) the first large field trial in the USA and/or Canada reporting a null or negative result on an efficacy endpoint. We calculated the median and interquartile ranges of the forecasts to measure both the consensus and range of opinions.

Finally, our survey collected basic demographic information and two Likert scale questions measuring comfort answering questions about the US and Canadian health care system and the extent to which experts think the pace of COVID-19 research will lead to more errors in the vaccine development process.

RESULTS

Our sample consisted of 28 experts with 25 median years of experience working with vaccines (range 8–42); 20 worked in academia, 6 in industry, and 2 in government. Participants were mostly based in Canada (20) and the USA (6); participants worked in academia (20), industry (6), and public health (2). Participants indicated they were very comfortable (46% or 13/28) or somewhat comfortable (46% or 13/28) answering questions about vaccine development in the USA or Canada.

The median and interquartile range of forecasters’ best guesses, soonest occurrence, and latest occurrence for each milestone are indicated in Figure 1. See Figure 2 for histograms of setback probabilities.

Figure 1
figure1

Boxplots of the best guess, soonest estimate, and latest estimate for each of the vaccine development milestones. The width of each box represents the interquartile range and the bar represents the median. The median best guess, soonest guess, and latest guess for the first field trial reporting results were respectively March 2021, December 2020, and July 2021 (interquartile ranges were respectively January 2021–April/May 2021, November 2020–January 2021, and June 2021–January 2022). The median best guess, soonest guess, and latest guess for a vaccine being available to at-risk individuals were respectively March/April 2021, February 2020, and December 2021 (interquartile ranges were respectively February 2021–September 2021, January 2021–April/May 2021, and June 2021–May 2022). The median best guess, soonest guess, and latest guess for the vaccine being available to the public were respectively September/October 2021, June 2021, and July 2022 (interquartile ranges were respectively June 2021–January 2022, March 2021–September 2021, and December 2021–October/November 2023).

Figure 2
figure2

Histograms of probabilities provided for the two setback questions. The median forecast that the FDA would issue a black box warning for a widely deployed vaccine was 30% (interquartile range 15–40%). For the probability of the first field study in the USA or Canada reporting a null or negative result on an efficacy endpoint, the median probability was 40% (interquartile range 23–51%).

Experts were split on whether the pace of COVID-19 research is increasing the potential for errors; 53% (15/28) agreed, 14% (4/28) were neutral, and 32% (9/28) disagreed.

DISCUSSION

Experts in our survey offered forecasts on vaccine development that were generally less optimistic than the timeline of early 2021 offered by US public officials.4 Experts seem to believe that a publicly available vaccine next summer is the best case scenario with the possibility that it may take until 2022. In addition, the median probabilities that a vaccine would receive a warning label and that a field study might not report positive outcomes were 30% and 40% respectively.

Our survey has important limitations. First, our sample is predominantly Canadian, though it is worth noting Canada has a productive vaccine development enterprise.5 Second, our elicitation approach was relatively simple to minimize survey burden for vaccine developers. Nevertheless, our estimates provide a more complete picture of expert belief than single point estimates or media quotes from individuals.

References

  1. 1.

    Coronavirus Disease (COVID-19) Situation Reports. Accessed July 15, 2020. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports

  2. 2.

    Morgan MG. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc Natl Acad Sci U S A. 2014;111(20):7176-7184. https://doi.org/10.1073/pnas.1319946111

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  3. 3.

    Davis-Stober C, Budescu D, Dana J, Broomell S. When is a crowd wise? Decision 2014; 1. https://doi.org/10.1037/dec0000004

  4. 4.

    Anthony Fauci on Covid-19 reopenings, vaccines, and “warp speed.” STAT. Published June 1, 2020. Accessed July 15, 2020. https://www.statnews.com/2020/06/01/anthony-fauci-on-covid-19-reopenings-vaccines-and-moving-at-warp-speed/

  5. 5.

    Wolf J, Bruno S, Eichberg M, et al. Applying lessons from the Ebola vaccine experience for SARS-CoV-2 and other epidemic pathogens. Npj Vaccines 2020;5(1):1-5. https://doi.org/10.1038/s41541-020-0204-7

    CAS  Article  Google Scholar 

Download references

Funding

This project was funded by a McGill Interdisciplinary Initiative in Infection and Immunity research grant.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jonathan Kimmelman.

Ethics declarations

Conflict of Interest

PBK, SBB, HM, AM, and JK declare no conflicts of interest. BJW serves as medical officer for Medicago Inc., a pre-clinical biotechnology company with a candidate vaccine for SARS-COV-2. JP has received consulting/honoraria fees from AbbVie, Cepheid and Seegene, and Jannsen and research grant funding outside of the current work from AbbVie, BD Diagnostics, Sanofi Pasteur, and MedImmune.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kane, P.B., Moyer, H., MacPherson, A. et al. Expert Forecasts of COVID-19 Vaccine Development Timelines. J GEN INTERN MED (2020). https://doi.org/10.1007/s11606-020-06244-9

Download citation