Problems with Early Systematic Reviews: The Case of Coronavirus Disease 2019 (COVID-19) in Pregnancy

Abstract

Introduction

Rapid dissemination of findings regarding the Coronavirus Disease 2019 (COVID-19) and its potential effects on pregnancy is crucial to support understanding and development of recommendations for optimization of obstetrics care. However, much of the current studies published are in the form of case reports or case series which can be prone to biases. Other factors also further complicate attempts to analyze data accurately. Hence, this evaluation hopes to highlight some of these problems and provide suggestions to help clinicians mitigate and make reasonable conclusions when reading the abundant yet limited body of evidence when furthering their research efforts.

Methods

Studies regarding COVID-19 and pregnancy were searched on databases such as PubMed, EMBASE, Scopus, the Cochrane Library. Manual search of references of select articles were also undertaken. Apart from summarizing study limitations identified by authors, the characteristics of current literature and systematic reviews were also evaluated to identify potential factors affecting accuracy of subsequent analysis.

Results

Factors such as innate biasness in study design of current literature, duplicate reporting, differing inclusion criteria of systematic reviews, scarce data, inadequate follow-up period and limitations of systematic reviews have been shown to hinder the ability for accurate data extrapolation.

Discussion

Unless additional studies are conducted in identified areas of data scarcity and a common list of factors affecting accuracy of data analysis are taken into account when developing recommendations, discrepancies will continue to arise and accurate data analysis and valid systematic reviews will be precluded.

Significance Statement

Information regarding COVID-19 and pregnancy are rapidly disseminated. However, the additional quantity of evidence does not translate to a similar improvement in the quality of evidence for systematic reviews as current evidence may be at risk of bias. Discrepancies arise when data is scarce and when different factors affecting biasness are omitted in data interpretation. To our knowledge, no current studies have specifically addressed factors affecting the accuracy and value of data analysis. This paper identifies some of these factors in order to provide shared knowledge of areas where caution should be exercised when interpreting such findings.

Introduction

Despite ongoing research efforts over the last eight months, much remains unknown about the Coronavirus Disease 2019 (COVID-19). Learning about COVID-19 infection and its potential effects on pregnancy and perinatal outcomes is crucial to guide obstetrical management during this disease outbreak. As such, many obstetric units have been expeditiously publishing data from their local studies in an attempt to rapidly disseminate information on COVID-19 in pregnancy. To date, most of these publications have been in the form of case reports and case series. While these reports provide valuable information which have helped to guide early management in this rapidly evolving global pandemic, they are not ranked highly in the traditional hierarchy of evidence (Royal College of Obstetricians and Gynaecologists 2020). Discrepancies in recommendations especially in areas with scarce data thus arise. This commentary aims to highlight some of the potential problems and limitations one may encounter when reviewing the existing literature.

Methods

A search was done on PubMed, EMBASE, Scopus and the Cochrane Library electronic databases. References of select articles for studies describing COVID-19 and pregnancy were also screened. This manuscript is not based upon clinical study or patient data. Since December 2019 when the infection was first identified, more than 41,000 MEDLINE-indexed papers have been published on COVID-19. Up to 31st July 2020, at least 29 systematic reviews on COVID-19 and pregnancy with varying conclusions have been published (Abdollahpour and Khadivzadeh 2020; Akhtar et al. 2020; Ashraf et al. 2020; Capobianco et al. 2020; Della Gatta et al. 2020; Deniz and Tezer 2020; Di Mascio et al. 2020; Dubey et al. 2020; Elshafeey et al. 2020; Hessami et al. 2020; Juan et al. 2020; Kasraeian et al. 2020; Kotlyar et al. 2020; Lopes de Sousa et al. 2020; Matar et al. 2020; Melo and Araújo 2020; Muhidin et al. 2020; Segars et al. 2020; Silva et al. 2020; Smith et al. 2020; Thomas et al. 2020a, b; Trippella et al. 2020; Trocado et al. 2020; Turan et al. 2020; Walker et al. 2020; Yang et al. 2020; Yoon et al. 2020; Zaigham and Andersson 2020). Factors which are likely to cause bias and thus, inaccuracy in subsequent analysis in systematic reviews were then identified and compiled.

Results

All the systematic reviews included in our analysis acknowledge the limitation that their review is limited to largely case series and case reports. Apart from the innate bias of case series and case reports, the current literature is complicated by inconsistent reporting and small sample size which further impede efforts to apply the findings to the larger population (Ashraf et al. 2020; Capobianco et al. 2020; Della Gatta et al. 2020; Di Mascio et al. 2020; Dubey et al. 2020; Hessami et al. 2020; Kotlyar et al. 2020; Lopes de Sousa et al. 2020; Matar et al. 2020; Sayre et al. 2017; Smith et al. 2020; Thomas et al. 2020a, b; Trippella et al. 2020; Trocado et al. 2020; Turan et al. 2020; Walker et al. 2020; Yang et al. 2020; Yoon et al. 2020; Zaigham et al. 2020). Duplicate reporting overemphasizes findings. They can occur when overlaps in patient series are not clearly stated, when the same case is reported by both the admitting and receiving hospitals or when different aspects of the same case are reported, making it difficult for screening by direct comparison of the clinical findings (Di Mascio et al. 2020; Elshafeey et al. 2020; Juan et al. 2020). Additionally, variations in management and healthcare resources exists between countries (Turan et al. 2020). Genetic and environmental factors can also influence the natural history of disease (Capobianco et al. 2020) and thus, these factors limit generalizability of the findings of systematic reviews (Dubey et al. 2020). Exclusion of studies not published in the English language may further cause bias (Matar et al. 2020; Muhidin et al. 2020; Thomas et al. 2020a, b).

Discrepancies arise when differing recommendations are made based on researchers’ personal experiences, synthesis of limited COVID-19 studies or studies done during previous disease outbreaks. Factors identified include the study design of existing literature, duplicate reporting, inadequate follow-up period and limitations of systematic reviews.

In most studies, the short follow-up period and inclusion of predominantly patients in their third trimester of pregnancy, might lead to an overestimation of risks such as preterm birth whilst underestimating longitudinal risks such as fetal growth restriction (Di Mascio et al. 2020; Thomas et al. 2020a, b; Yang et al. 2020). Increased risk of preterm birth may be confounded by other factors such as the socioeconomic characteristics of patients, decisions to expedite delivery to optimize maternal condition and psychological stress during pregnancy. (Melo et al. 2020) In reports where indication for delivery was not clearly specified, extrapolation of data about the rate of spontaneous versus iatrogenic preterm birth was near-impossible. (Kasraeian et al. 2020) Hence, the lack of comparable controls negates attributing these complications and outcomes solely to COVID-19 infection. (Trocado et al. 2020) Moreover, most studies included only patients with laboratory confirmed positive quantitative reverse transcriptase polymerase chain reaction (qRT-PCR) assay. However, the viral nucleic acid test has a false-negative rate of up to 30% (Calda et al. 2020). Asymptomatic carriers of COVID-19 were only picked up incidentally during universal screening (Breslin et al. 2020). Thus, the actual prevalence of COVID-19 amongst pregnant women, and the associated COVID-19 pregnancy outcomes and complications might be underreported (Kotlyar et al. 2020; Matar et al. 2020).

In the face of an evolving disease, systematic reviews are only accurate and updated up to the point of submission for publication (Elshafeey et al. 2020; Segars et al. 2020). Additionally, the study protocol of systematic reviews typically excludes papers published in the format of commentaries, opinions, expert reviews, and letters to editors. Therefore, important insights from these studies might not be captured in systematic reviews. Evaluation of the quality of included studies must not be missed, even if performed during a pandemic, to ensure readers are not misled about the findings of the systematic review (Capobianco et al. 2020).

Discussion

Rapid dissemination of information regarding COVID-19 is necessary for clinicians to keep abreast of the latest developments, thereby guiding and updating clinical management as deemed appropriate. However, reviewers should bear in mind the limitations and interpret findings with caution. At present, the conclusions drawn from the current body of evidence may be at risk of bias.

To mitigate biasness, better quality evidence from large laboratories and standardized registries are important. In the absence of such databases, strict study protocol should be implemented, with special attention paid to studies originating from countries with high COVID-19 related research output. Recognition of the admitting and receiving hospitals might also help to identify the possibility of duplicate reporting. Establishment of carefully governed national and international registries or an international standard for publication of data can help to address some of these issues.

Nonetheless, we are immensely grateful for the tremendous ongoing research efforts during this challenging time of a global pandemic. As we continue the fight against COVID-19, we look forward to more rigorous research studies and higher level of evidence which will help us more reliably better our understanding in managing COVID-19 in pregnancy.

References

  1. Abdollahpour, S., & Khadivzadeh, T. (2020). Improving the quality of care in pregnancy and childbirth with coronavirus (COVID-19): a systematic review. Journal of Maternal-Fetal and Neonatal Medicine, 1–9. https://doi.org/10.1080/14767058.2020.1759540

  2. Akhtar, H., Patel, C., Abuelgasim, E., & Harky, A. (2020). COVID-19 (SARS-CoV-2) Infection in pregnancy: A systematic reviewGynecologic and Obstetric Investigation, 1–12.https://doi.org/10.1159/000509290

  3. Ashraf, M. A., Keshavarz, P., Hosseinpour, P., Erfani, A., Roshanshad, A., Pourdast, A., … Poordast, T. (2020). Coronavirus disease 2019 (COVID-19): A systematic review of pregnancy and the possibility of vertical transmission. The Journal of Reproduction & Infertility, 21(3), 157–168.

  4. Breslin, N., Baptiste, C., Gyamfi-Bannerman, C., Miller, R., Martinez, R., Bernstein, K., … Goffman, D. (2020). COVID-19 infection among asymptomatic and symptomatic pregnant women: Two weeks of confirmed presentations to an affiliated pair of New York City hospitals. American Journal of Obstetrics & Gynecology MFM, 100118https://doi.org/10.1016/j.ajogmf.2020.100118

  5. Calda P, B. M., Fischerová D, Zikán M, Smetanová D, & Machala L. (2020). Coronavirus infection and pregnancy (COVID-19). Opinion of the Czech Society for Ultrasound in Obstetrics and Gynecology. Actual Gynecology, 12, 17–19.

  6. Capobianco, G., Saderi, L., Aliberti, S., Mondoni, M., Piana, A., Dessole, F., … Sotgiu, G. (2020). COVID-19 in pregnant women: A systematic review and meta-analysis. European Journal of Obstetrics & Gynecology and Reproductive Biology, 252, 543–558. https://doi.org/10.1016/j.ejogrb.2020.07.006

  7. Della Gatta, A. N., Rizzo, R., Pilu, G., & Simonazzi, G. (2020). Coronavirus disease 2019 during pregnancy: A systematic review of reported cases. American Journal of Obstetrics and Gynecology. https://doi.org/10.1016/j.ajog.2020.04.013.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Deniz, M., & Tezer, H. (2020). Vertical transmission of SARS CoV-2: A systematic reviewJournal of Maternal-Fetal and Neonatal Medicine, 1–8.https://doi.org/10.1080/14767058.2020.1793322

  9. Di Mascio, D., Khalil, A., Saccone, G., Rizzo, G., Buca, D., Liberati, M., … D’Antonio, F. (2020). Outcome of Coronavirus spectrum infections (SARS, MERS, COVID 1 -19) during pregnancy: A systematic review and meta-analysis. American Journal of Obstetrics and Gynecology MFM, 100107. https://doi.org/10.1016/j.ajogmf.2020.100107

  10. Dubey, P., Reddy, S. Y., Manuel, S., & Dwivedi, A. K. (2020). Maternal and neonatal characteristics and outcomes among COVID-19 infected women: An updated systematic review and meta-analysis. European Journal of Obstetrics, Gynecology, and Reproductive Biology, 252, 490–501. https://doi.org/10.1016/j.ejogrb.2020.07.034.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  11. Elshafeey, F., Magdi, R., Hindi, N., Elshebiny, M., Farrag, N., Mahdy, S., … Nabhan, A. (2020). A systematic scoping review of COVID-19 during pregnancy and childbirth. International Journal of Gynecology & Obstetrics, 150(1), 47–52https://doi.org/10.1002/ijgo.13182

  12. Hessami, K., Homayoon, N., Hashemi, A., Vafaei, H., Kasraeian, M., & Asadi, N. (2020). COVID-19 and maternal, fetal and neonatal mortality: A systematic review.Journal of Maternal-Fetal and Neonatal Medicine, 1–6https://doi.org/10.1080/14767058.2020.1806817

  13. Juan, J., Gil, M. M., Rong, Z., Zhang, Y., Yang, H., & Poon, L. C. (2020). Effects of coronavirus disease 2019 (COVID-19) on maternal, perinatal and neonatal outcomes: A systematic review. Ultrasound in Obstetrics and Gynecology. https://doi.org/10.1002/uog.22088.

    Article  PubMed  Google Scholar 

  14. Kasraeian, M., Zare, M., Vafaei, H., Asadi, N., Faraji, A., Bazrafshan, K., & Roozmeh, S. (2020). COVID-19 pneumonia and pregnancy; A systematic review and meta-analysisJournal of Maternal-Fetal and Neonatal Medicine, 1–8https://doi.org/10.1080/14767058.2020.1763952

  15. Kotlyar, A., Grechukhina, O., Chen, A., Popkhadze, S., Grimshaw, A., Tal, O., … Tal, R. (2020). Vertical transmission of COVID-19: A systematic review and meta-analysis. American Journal of Obstetrics and Gynecology.https://doi.org/10.1016/j.ajog.2020.07.049

  16. Lopes de Sousa, Á. F., de Carvalho, H. E.F., de Oliveira, L. B., Schneider, G., Camargo, E. L. S., & Watanabe, E, de Andrade, D., Fernandes, A. F. C., Mendes, I. A. C., & Fronteira, I. (2020). Effects of COVID-19 infection during pregnancy and neonatal prognosis: What is the evidence? International Journal of Environmental Research and Public Health, 17(4176), 36.https://doi.org/10.3390/ijerph17114176

  17. Matar, R., Alrahmani, L., Monzer, N., Debiane, L. G., Berbari, E., Fares, J., … Murad, M. H. (2020). Clinical presentation and outcomes of pregnant women with COVID-19: A systematic review and meta-analysis. Clinical Infectious Diseases. https://doi.org/10.1093/cid/ciaa828

  18. Melo, G. C., & Araújo, K. (2020). COVID-19 infection in pregnant women, preterm delivery, birth weight, and vertical transmission: A systematic review and meta-analysis. Cad Saude Publica, 36(7), e00087320. https://doi.org/10.1590/0102-311x00087320.

    Article  PubMed  Google Scholar 

  19. Muhidin, S., Behboodi Moghadam, Z., & Vizheh, M. (2020). Analysis of maternal coronavirus infections and neonates born to mothers with 2019-nCoV; A systematic review. Archives of Academic Emergency Medicine, 8(1), e49.

    PubMed  PubMed Central  Google Scholar 

  20. Royal College of Obstetricians and Gynaecologists. ( 2020). Developing a Green-top guideline. Retrieved from https://www.rcog.org.uk/globalassets/documents/guidelines/rcog-guideline-development-guide.pdf

  21. Sayre, J. W., Toklu, H. Z., Ye, F., Mazza, J., & Yale, S. (2017). Case reports, case series – From clinical practice to evidence-based medicine in graduate medical education. Cureus, 9(8), e1546. https://doi.org/10.7759/cureus.1546.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Segars, J., Katler, Q., McQueen, D. B., Kotlyar, A., Glenn, T., Knight, Z., … Kawwass, J. F. (2020). Prior and novel coronaviruses, Coronavirus Disease 2019 (COVID-19), and human reproduction: what is known? Fertil Steril, 113(6), 1140–1149. https://doi.org/10.1016/j.fertnstert.2020.04.025

  23. Silva, C., Oliveira, L. V., Lopes, L. P., Santos, W., & Agra, I. K. R. (2020). Immunological aspects of coronavirus disease during pregnancy: An integrative review. Revista da Associação Médica Brasileira (1992), 66(5), 696–700. https://doi.org/10.1590/1806-9282.66.5.696

  24. Smith, V., Seo, D., Warty, R., Payne, O., Salih, M., Chin, K. L., … Wallace, E. (2020). Maternal and neonatal outcomes associated with COVID-19 infection: A systematic review. PLoS One, 15(6), e0234187. https://doi.org/10.1371/journal.pone.0234187

  25. Thomas, B., Pallivalapila, A., El Kassem, W., Tarannum, A., Al Hail, F., Rijims, M., … Al Hail, M. (2020). Maternal and perinatal outcomes and pharmacological management of Covid-19 infection in pregnancy: A systematic review protocol. Systematic Reviews, 9(1), 161.https://doi.org/10.1186/s13643-020-01418-2

  26. Thomas, P., Alexander, P. E., Ahmed, U., Elderhorst, E., El-Khechen, H., Mammen, M. J., … Alhazzani, W. (2020). Vertical transmission risk of SARS-CoV-2 infection in the third trimester: A systematic scoping review. Journal of Maternal-Fetal and Neonatal Medicine, 1–8. https://doi.org/10.1080/14767058.2020.1786055

  27. Trippella, G., Ciarcià, M., Ferrari, M., Buzzatti, C., Maccora, I., Azzari, C., … Chiappini, E. (2020). COVID-19 in pregnant women and neonates: A systematic review of the literature with quality assessment of the studies. Pathogens, 9(6). https://doi.org/10.3390/pathogens9060485

  28. Trocado, V., Silvestre-Machado, J., Azevedo, L., Miranda, A., & Nogueira-Silva, C. (2020). Pregnancy and COVID-19: A systematic review of maternal, obstetric and neonatal outcomes.Journal of Maternal-Fetal and Neonatal Medicine, 1–13. https://doi.org/10.1080/14767058.2020.1781809

  29. Turan, O., Hakim, A., Dashraath, P., Jeslyn, W. J. L., Wright, A., & Abdul Kadir, R. (2020). Clinical characteristics, prognostic factors, and maternal and neonatal outcomes of SARS-CoV-2 infection among hospitalized pregnant women: A systematic review. International Journal of Gynaecology and Obstetrics. https://doi.org/10.1002/ijgo.13329.

    Article  PubMed  Google Scholar 

  30. Walker, K. F., O’Donoghue, K., Grace, N., Dorling, J., Comeau, J. L., Li, W., & Thornton, J. G. (2020). Maternal transmission of SARS-COV-2 to the neonate, and possible routes for such transmission: A systematic review and critical analysis. BJOG. https://doi.org/10.1111/1471-0528.16362.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Yang, Z., Wang, M., Zhu, Z., & Liu, Y. (2020). Coronavirus disease 2019 (COVID-19) and pregnancy: A systematic review. Journal of Maternal-Fetal and Neonatal Medicine, 1–4.https://doi.org/10.1080/14767058.2020.1759541

  32. Yoon, S. H., Kang, J. M., & Ahn, J. G. (2020). Clinical outcomes of 201 neonates born to mothers with COVID-19: A systematic review. European Review for Medical and Pharmacological Sciences, 24(14), 7804–7815. https://doi.org/10.26355/eurrev_202007_22285.

    CAS  Article  PubMed  Google Scholar 

  33. Zaigham, M., & Andersson, O. (2020). Maternal and perinatal outcomes with COVID-19: A systematic review of 108 pregnancies. Acta Obstetricia et Gynecologica Scandinavica, 99(7), 823–829. https://doi.org/10.1111/aogs.13867.

    CAS  Article  PubMed  Google Scholar 

Download references

Funding

No funding or sponsorships were received for this study or publication of this article.

Author information

Affiliations

Authors

Contributions

XLA: Preparation, manuscript original writing and editing. SPC, MSQC and SS: supervision, manuscript review and edit. JCSL: Conceptualization, supervision, manuscript review and edit.

Corresponding author

Correspondence to Jill Cheng Sim Lee.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest. All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.

Ethical Approval

No ethics approval applicable for this commentary.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ang, X.L., Chonkar, S.P., Chua, M.S.Q. et al. Problems with Early Systematic Reviews: The Case of Coronavirus Disease 2019 (COVID-19) in Pregnancy. Matern Child Health J (2020). https://doi.org/10.1007/s10995-020-03046-7

Download citation

Keywords

  • COVID-19
  • Novel coronavirus 2019
  • Pregnancy
  • Pandemic
  • Systematic reviews