Skip to main content
SpringerLink
Log in

Systematic Reviews and Meta-Analyses of Non-randomised Studies

  • Chapter
  • First Online:
Principles and Practice of Systematic Reviews and Meta-Analysis

Abstract

Randomised controlled trials (RCTs) are considered as the gold standard for clinical research because unlike other study designs, they control for known, and importantly, unknown confounders by randomisation. Evaluation of interventions should hence be ideally done by RCTs. However, RCTs are not always possible or feasible for various reasons, including ethical concerns and the need for time, effort, and funding. Difficulty in the generalisation of the findings of RCTs is also an issue given their rigid design. Non-randomised studies (non-RCTs) provide an alternative to RCTs in such situations. These include cohort, case-control and cross-sectional studies. Non-RCTs have the advantage of providing data from the real-life situation rather than that from the rigid framework of RCTs. The limitations of non-RCTs include selection bias and lack of randomisation that allow confounders to influence the results. At best, non-RCTs can only generate hypotheses for testing in RCTs. This chapter covers the methodology for conducting, reporting and interpreting systematic reviews and meta-analysis of non-RCTs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Abrahama NS, Byrneb CJ, Young JM, Solomon MJ. Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials. J Clin Epidemiol. 2010;63:238–45. https://doi.org/10.1016/j.jclinepi.2009.04.005.

    Article  Google Scholar 

  • Balasubramanian H, Ananthan A, Rao S, Patole S. Odds ratio vs risk ratio in randomised controlled trials. Postgrad Med. 2015;127(4):359–67.

    Article  Google Scholar 

  • Bland JM, Altman DG. The odds ratio. BMJ. 27 May 2000; 320: 1468.

    Google Scholar 

  • Szumilas M. Explaining odds ratios. Can Acad Child Adolesc Psychiatry. 2010; 19(3): 227–229.

    Google Scholar 

  • Concato J, Shah N, Horwitz RI. Randomised, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):1887–92.

    Article  CAS  Google Scholar 

  • Cummings P. The relative merits of risk ratios and odds ratios. Arch Pediatr Adolesc Med. 2009;163(5):438–45.

    Article  Google Scholar 

  • Deeks JJ, Dinnes J, D’Amico R, et al. International Stroke Trial Collaborative Group European Carotid Surgery Trial Collaborative Group. Evaluating non-randomised intervention studies. Health Technol Assess. 2003;7:iii–x, 1–173.

    Google Scholar 

  • Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52:377–84.

    Article  CAS  Google Scholar 

  • Faber T, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review. BMC Med Res Methodol. 2016;16:35. https://doi.org/10.1186/s12874-016-0136-0.

    Article  PubMed  PubMed Central  Google Scholar 

  • Faraoni D, Schaefer ST. Randomised controlled trials vs. observational studies: why not just live together? BMC Anesthesiol. 2016 Oct 21;16(1):102.

    Google Scholar 

  • Gershon AS, Jafarzadeh SR, Wilson KC, Walkey A. Clinical knowledge from observational studies: everything you wanted to know but were afraid to ask. Am J Respir Crit Care Med. 2018; 198 (7):859–867.

    Google Scholar 

  • Gerstein HC, McMurray J, Holman RR. Real-world studies no substitute for RCTs in establishing efficacy. Lancet. 2019;393:210–1.

    Article  Google Scholar 

  • Gilmartin Thomas JFM, Liew D. Observational studies and their utility for practice. Aust Prescr. 2018;41:82–5.

    Google Scholar 

  • Gueyffier F, Cucherat M. The limitations of observation studies for decision making regarding drugs efficacy and safety. Therapie. 2019;74:181–5.

    Article  Google Scholar 

  • Guyatt GH, Oxman AD, Santesso N, et al. GRADE guidelines: 12 Preparing summary of findings tables—binary outcomes. J Clin Epidemiol 2013;66:158–172.

    Google Scholar 

  • Hartling L, Milne A, Hamm MP, Vandermeer B, Ansari M, Tsertsvadze A, Dryden DM. Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers. J Clin Epidemiol. 2013;66:982–93.

    Article  Google Scholar 

  • Heikinheimo O, Bitzer J, Rodríguez LG. Real-world research and the role of observational data in the field of gynaecology–a practical review. Eur J Contracept Reprod Health Care. 2017;22(4):250–9.

    Article  Google Scholar 

  • Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol. 2005;5:13.

    Article  Google Scholar 

  • Jepsen P, Johnsen SP, Gillman MW, Sorensen HT. Interpretation of observational studies. Heart. 2004;90(8):956–60.

    Article  CAS  Google Scholar 

  • Joffe MM, Rosenbaum PR. Invited commentary: propensity scores. Am J Epidemiol. 1999;150(4):327–33.

    Article  CAS  Google Scholar 

  • Lau J, Ioannidis JP, Terrin N, et al. The case of the misleading funnel plot. BMJ. 2006;333:597–600.

    Article  Google Scholar 

  • Lefebvre C, Manheimer E, Glanville J. Searching for studies. Cochrane handbook for systematic reviews of interventions. New York: Wiley 2008:95–150.

    Google Scholar 

  • Lefebvre C, Glanville J, Wieland LS, et al. Methodological developments in searching for studies for systematic reviews: past, present and future? Syst Rev. 2013;2:78.

    Article  Google Scholar 

  • Ligthelm RJ, Borzi V, Gumprecht J, Kawamori R, Wenying Y, Valensi P. Importance of observational studies in clinical practice. Clin Ther. 2007;29 Spec No:1284–92.

    Google Scholar 

  • Luchini C, Stubbs B, Solmi M, Veronese N. Assessing the quality of studies in meta-analyses: Advantages and limitations of the Newcastle Ottawa Scale. World J Meta-Anal. Aug 26, 2017; 5(4): 80–84. Published online Aug 26, 2017. https://doi.org/10.13105/wjma.v5.i4.80.

  • Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med J. 2003;20(1):54–60.

    Google Scholar 

  • Mariani AW, Pego-Fernandes PM. Observational studies: why are they so important? Sao Paulo Med J. 2014;132(1):01–02 https://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516-31802014000100001&lng=en&tlng=en. Accessed 10 Aug 2020.

  • McHugh ML. The odds ratio: calculation, usage and interpretation. Biochemic Med. 2009;19 (2):120–126.

    Google Scholar 

  • Norris S, Atkins D, Bruening W, et al. Selecting observational studies for comparing medical interventions. In: Agency for Healthcare Research and Quality. Methods Guide for Comparative Effectiveness Reviews [posted June 2010]. Rockville, MD. http://www.effectivehealthcare.ahrq.gov/ehc/products/196/454/MethodsGuideNorris_06042010.pdf. Accessed 11 Aug 2020.

  • Oremus M, Oremus C, Hall GB, McKinnon MC; ECT & Cognition Systematic Review Team. Inter-rater and test-retest reliability of quality assessments by novice student raters using the Jadad and Newcastle-Ottawa Scales. BMJ Open. 2012;2:e001368.

    Google Scholar 

  • Reeves BC, Deeks JJ, Higgins JPT, et al. Chapter 24: Including non-randomised studies on intervention effects. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. www.training.cochrane.org/handbook. Accessed 10th Aug 2020.

  • Saroha V, Josephson CD, Patel RM. Epidemiology of necrotising enterocolitis: New considerations regarding the influence of red blood cell transfusions and anemia. Clin Perinatol. 2019;46(1):101–17. https://doi.org/10.1016/j.clp.2018.09.006.

    Article  PubMed  Google Scholar 

  • Schillaci G, Battista F, Pucci G. Are observational studies more informative than randomised controlled trials in hypertension? ConSide of the Argument. Hypertension. 2013;62:470–6.

    Article  CAS  Google Scholar 

  • Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358: https://doi.org/10.1136/bmj.j4008 (Published 21/9/2017).

    Article  PubMed  PubMed Central  Google Scholar 

  • Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.

    Google Scholar 

  • Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283:2008–12.

    Article  CAS  Google Scholar 

  • Veronese N, Carraro S, Bano G, Trevisan C, Solmi M, Luchini C, Manzato E, Caccialanza R, Sergi G, Nicetto D. Hyperuricemia protects against low bone mineral density, osteoporosis and fractures: a systematic review and meta-analysis. Eur J Clin Invest. 2016;46:920–30.

    Article  CAS  Google Scholar 

  • Wan X, Wang W, Liu J, et al. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14:135.

    Article  Google Scholar 

  • Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality if non-randomised studies in meta-analyses, 2012. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.

Download references

Author information

Authors and Affiliations

  1. School of Medicine, University of Western Australia, Perth, WA, 6009, Australia

    Sanjay Patole

  2. Neonatal Directorate, King Edward Memorial Hospital for Women, Perth, WA, 6008, Australia

    Sanjay Patole

Authors
  1. Sanjay Patole
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanjay Patole .

Editor information

Editors and Affiliations

  1. School of Medicine, University of Western Australia, Perth, WA, Australia

    Sanjay Patole

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Patole, S. (2021). Systematic Reviews and Meta-Analyses of Non-randomised Studies. In: Patole, S. (eds) Principles and Practice of Systematic Reviews and Meta-Analysis. Springer, Cham. https://doi.org/10.1007/978-3-030-71921-0_13

Download citation

Publish with us

Policies and ethics

Search

Navigation