Skip to main content

Steps in the undertaking of a systematic review in orthopaedic surgery

International Orthopaedics volume 36, pages 477–484 (2012)Cite this article

Abstract

In the last decades of the twentieth century it became obvious that modern medical care is replete with data and information, but in need of reliable evidence. This has led to an increased effort to systematically synthesise medical research and make it more useful for practitioners. Systematic reviews use an approach to research synthesis that minimises the risk of misinterpreting a body of evidence due to incomprehensive search or subjective opinion. Carrying out a systematic review is a rigorous procedure which corresponds to standard methodological steps in primary research studies. It involves posing a well-defined question, developing a robust search strategy, screening for relevant primary studies, critical appraisal of included studies, data extraction and processing, analysis and interpretation of results. In some, but not all systematic reviews it is appropriate to conduct a meta-analysis, which is a statistical procedure that integrates the results of several independent studies. Results of meta-analysis are graphically presented in forest plots, with pooled point estimate and its confidence interval represented as a rhombus, usually called a “diamond”. Methodological quality of systematic reviews should not be judged by the quality of primary studies included, but by a distinct set of criteria specified in assessment tools such as AMSTAR. Systematic reviews and meta-analyses should be reported according to the PRISMA checklist. A major contribution to the development of methodological standards has been given by The Cochrane Collaboration, whose Handbook of Systematic Reviews of Interventions is the primary reference for all authors and referees of systematic reviews in health care.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (1996) Evidence based medicine: what it is and what it isn’t. BMJ 312:71–72

    PubMed  Article  CAS  Google Scholar 

  2. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (1992) A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 268:240–248

    PubMed  Article  CAS  Google Scholar 

  3. Ernst E, Pittler MH (2001) Assessment of therapeutic safety in systematic reviews: literature review. BMJ 323:546

    PubMed  Article  CAS  Google Scholar 

  4. Lane S (2009) Sense about systematic reviews. Available via http://www.senseaboutscience.org/data/files/resources/52/Sense-About-Systematic-Reviews.pdf

  5. Wale J, Colombo C, Belizan M, Nadel J (2010) International health consumers in the Cochrane Collaboration: fifteen years on. J Ambul Care Manage 33:182–189

    PubMed  Google Scholar 

  6. Wale JL, Belizán M, Nadel J, Jeffrey C, Vij SL (2011) The Cochrane Library review titles that are important to users of health care, a Cochrane Consumer Network project. Health Expect [Epub ahead of print]. doi: 10.1111/j.1369-7625.2011.00723.x

  7. Straus SE, Richardson WS, Glasziou P, Haynes RB (2005) Evidence-based medicine: how to practice and teach EBM, 3rd edn. Churchill Livingstone, Edinburgh

    Google Scholar 

  8. Oxford Centre for Evidence-based Medicine (2001) Levels of evidence. http://www.cebm.net/index.aspx?o=1047. Accessed 14 Nov 2011

  9. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J et al (2011) GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 64:401–406. doi:dx.doi.org

    PubMed  Article  Google Scholar 

  10. Reeves BC, Deeks JJ, Higgins JPT, Wells GA, on behalf of the Cochrane Non-Randomised Studies Methods Group (2011) Chapter 13: Including non-randomized studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011

  11. Clifton R, Haleem S, McKee A, Parker MJ (2008) Closed suction surgical wound drainage after hip fracture surgery: a systematic review and meta-analysis of randomised controlled trials. Int Orthop 32:723–727

    PubMed  Article  CAS  Google Scholar 

  12. Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C (2000) Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials 21:476–487

    PubMed  Article  CAS  Google Scholar 

  13. Lefebvre C, Manheimer E, Glanville J, on behalf of the Cochrane Information Retrieval Methods Group (2011) Chapter 6: Searching for studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011

  14. Mead TL, Richards DT (1995) Librarian participation in meta-analysis projects. Bull Med Libr Assoc 83:461–464

    PubMed  CAS  Google Scholar 

  15. Bates MJ (1989) The design of browsing and berrypicking techniques for the online search interface. Online Review 13:407–424

    Article  Google Scholar 

  16. Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F et al (2007) Clinical trial registration: looking back and moving ahead. JAMA 298:93–94

    PubMed  Article  CAS  Google Scholar 

  17. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K (2009) Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 1:MR000006

  18. Ibrahim T, Tleyjeh IM, Gabbar O (2008) Surgical versus non-surgical treatment of chronic low back pain: a meta-analysis of randomised trials. Int Orthop 32:107–113

    PubMed  Article  CAS  Google Scholar 

  19. Bero LA, Grilli R, Grimshaw J, Harvey E, Oxman AD (1997) External refereeing of protocols for systematic reviews [abstract]. Prague, Czech Republic, Third International Congress on Biomedical Peer Review and Global Communications

  20. Silagy CA, Middleton P, Hopewell S (2002) Publishing protocols of systematic reviews. Comparing what was done to what was planned. JAMA 287:2831–2834

    PubMed  Article  Google Scholar 

  21. Kirkham JJ, Altman DG, Williamson PR (2010) Bias due to changes in specified outcomes during the systematic review process. PLoS One 5(3):e9810

    PubMed  Article  Google Scholar 

  22. Glasziou P, Sanders S, Pirozzo S, Doust J, Pietrzak E (2002) Abstract screening—the value of two reviewers [abstract]. Pushing the Boundaries, Oxford, United Kingdom, Fourth Symposium on Systematic Reviews

  23. Stavlas P, Roberts CS, Xypnitos FN, Giannoudis PV (2010) The role of reduction and internal fixation of Lisfranc fracture-dislocations: a systematic review of the literature. Int Orthop 34:1083–1091

    PubMed  Article  Google Scholar 

  24. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP (2006) Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 59:697–703

    PubMed  Article  Google Scholar 

  25. Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C et al (2006) Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 185:263–267

    PubMed  Google Scholar 

  26. Higgins JPT, Deeks JJ (2011) Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration

  27. Cheng T, Feng JG, Liu T, Zhang XL (2009) Minimally invasive total hip arthroplasty: a systematic review. Int Orthop 33:1473–1481

    PubMed  Article  CAS  Google Scholar 

  28. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S (1995) Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 16:62–73

    PubMed  Article  CAS  Google Scholar 

  29. Higgins JPT, Altman DG, Sterne JAC on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group (2011) Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011

  30. Xue D, Zheng Q, Li H, Qian S, Zhang B, Pan Z (2011) Selective COX-2 inhibitor versus nonselective COX-1 and COX-2 inhibitor in the prevention of heterotopic ossification after total hip arthroplasty: a meta-analysis of randomised trials. Int Orthop 35:3–8

    PubMed  Article  Google Scholar 

  31. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, for the QUOROM Group (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement. Lancet 354:1896–1900

    PubMed  Article  CAS  Google Scholar 

  32. Moher D, Liberati A, Tetzlaff J, Altman DG for the PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097

  33. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6:e1000100

    PubMed  Article  Google Scholar 

  34. Lewis S, Clarke M (2001) Forest plots: trying to see the wood and the trees. BMJ 322:1479–1480

    PubMed  Article  CAS  Google Scholar 

  35. Liu M, Yang Z, Pei F, Huang F, Chen S, Xiang Z (2010) A meta-analysis of the Gamma nail and dynamic hip screw in treating peritrochanteric fractures. Int Orthop 34:323–328

    PubMed  Article  Google Scholar 

  36. Furlan A, Clarke J, Esmail R, Sinclair S, Irvin E, Bombardier C (2001) A critical review of reviews on the treatment of chronic low back pain. Spine 26:E155–E162

    PubMed  Article  CAS  Google Scholar 

  37. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG (2007) Epidemiology and reporting characteristics of systematic reviews. PLoS Med 4:e78

    PubMed  Article  Google Scholar 

  38. Shea BJ, Dubé C, Moher D (2001) Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools. In: Egger M, Smith GD, Altman DG (eds) Systematic reviews in health care: meta-analysis in context. BMJ Books, London, pp 122–139

    Chapter  Google Scholar 

  39. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C et al (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10

    PubMed  Article  Google Scholar 

  40. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J et al (2009) AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 62:1013–1020

    PubMed  Article  Google Scholar 

  41. Eden J, Levit L, Berg A, Morton S (eds) (2011) Finding what works in health care: standards for systematic reviews. National Academies Press, Washington, DC

    Google Scholar 

  42. Starr M, Chalmers I, Clarke M, Oxman AD (2009) The origins, evolution, and future of The Cochrane Database of Systematic Reviews. Int J Technol Assess Health Care 25(1):182–195

    PubMed  Article  Google Scholar 

  43. Higgins JPT, Green S, eds (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011

  44. Lang TA (2004) The value of systematic reviews as research activities in medical education. Acad Med 79:1067–1072

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

Dario Sambunjak is financially supported by the Croatian Ministry of Science, Education and Sports (grant No. 216-1080314-0245 to Matko Marusic).

Author information

Affiliations

  1. Department of Research in Biomedicine and Health, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia

    Dario Sambunjak

  2. Croatian Branch of the Italian Cochrane Centre, Split, Croatia

    Dario Sambunjak

  3. Department of Orthopaedic Surgery, Dubrava University Hospital, Zagreb, Croatia

    Miljenko Franić

Authors
  1. Dario Sambunjak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miljenko Franić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dario Sambunjak.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Sambunjak, D., Franić, M. Steps in the undertaking of a systematic review in orthopaedic surgery. International Orthopaedics (SICOT) 36, 477–484 (2012). https://doi.org/10.1007/s00264-011-1460-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00264-011-1460-y

Keywords

  • Systematic Review
  • Review Author
  • Review Question
  • Review Finding
  • International Orthopaedic