Skip to main content
SpringerLink
Log in

The quality of guidelines in pediatric surgery: can we all AGREE?

  • Original Article
  • Published:
Pediatric Surgery International Aims and scope Submit manuscript

Abstract

Objective

Guidelines are meant to facilitate evidence-based clinical decision-making but vary in methodological rigor and quality of reporting. We assessed the quality of guidelines published in major pediatric surgery journals.

Methods

A MEDLINE search of 4 key pediatric surgery journals was performed. Included studies had guidelines, clinical practice guidelines, and consensus statements as a subject heading or keyword. Evaluations of guidelines were excluded. Eligible guidelines were assessed by three reviewers using the Appraisal of Guidelines for Research and Evaluation (AGREE II) Instrument.

Results

Our search identified ten guidelines for review. Agreement for study selection was excellent [K = 0.81 (95 % CI 0.63–0.99)]. The mean AGREE II score for individual guidelines was 18 % (SD 5.7 %). The best-scored quality domains were “scope and purpose” [mean score 49 % (SD 8.7 %)] and “clarity of presentation” [mean score 40 % (SD 18.7 %)]. The poorest score was for “editorial independence” [mean score 2 % (SD 3.7 %)].

Conclusions

The overall quality of guidelines in pediatric surgery, using AGREE II, is poor and may lead to inappropriate clinical decisions. Increased awareness of proper reporting and the methodological requirements for guideline development are needed to optimize the potential of guideline recommendations to improve practice.

Level of evidence

n/a (Quality Appraisal)

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Sackett DL, Rosenberg WM, Gray JA et al (1996) Evidence based medicine: what it is and what it isn’t. Br Med J 312:71–72

    Article  CAS  Google Scholar 

  2. Wilson M, Lavis J, Grimshaw J (2012) Supporting the use of research evidence in the canadian health sector. Healthc Q 15:58–62

    Article  PubMed  Google Scholar 

  3. Hayward RSA, Wilson MC, Tunis SR et al (1997) Practice guidelines: what are internists looking for? J Gen Intern Med 11:176–178

    Article  Google Scholar 

  4. Graham R, Mancher M, Wolman DM et al (eds) (2011) Clinical practice guidelines we can trust: committee on standards for developing trustworthy clinical practice guidelines. National Academies Press, Washington, DC

    Google Scholar 

  5. Ellrodt G, Cook DJ, Lee J et al (1997) Evidence-based disease management. JAMA 278:1687–1692

    Article  CAS  PubMed  Google Scholar 

  6. Rangel SJ, Kelsey J, Colby CE et al (2003) Development of a quality assessment scale for retrospective clinical studies in pediatric surgery. J Pediatr Surg 38:390–396

    Article  PubMed  Google Scholar 

  7. http://www.jpedsurg.org. Accessed 25 Aug 2014

  8. Wynne KE, Simpson BJ, Berman L et al (2011) Results of a longitudinal study of rigorous manuscript submission guidelines designed to improve the quality of clinical research reporting in a peer-reviewed surgical journal. J Pediatr Surg 46:131–137

    Article  PubMed  Google Scholar 

  9. Moher D, Liberati A, Tetzlaff J, The PRISMA Group et al (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339:332–336

    Article  Google Scholar 

  10. von Elm E, Altman DG, Egger M, STROBE Initiative et al (2008) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol 61:344–349

    Article  Google Scholar 

  11. Schulz KF, Altman DG, Moher D (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. The CONSORT Group. BMJ 340:698–702

    Article  Google Scholar 

  12. The AGREE Collaboration (2003) Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care 12:18–23

    Article  Google Scholar 

  13. Brouwers MC, Kho ME, Browman GP et al (2010) The Global Rating Scale complements the AGREE II in advancing the quality of practice guidelines. J Clin Epidemiol 65:526–534

    Article  Google Scholar 

  14. Goldin AB, LaRiviere C, Arca MJ, 2010 APSA Outcomes and Clinical Trials Committee et al (2012) Guidelines for surveys of the American Pediatric Surgical Association. J Pediatr Surg 46:2012–2017

    Article  Google Scholar 

  15. The AGREE Collaboration: AGREE instrument 2001 http://www.agreecollaboration.org/pdf/agreeinstrumentfinal.pdf. Accessed 24 Aug 2014

  16. Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46

    Article  Google Scholar 

  17. Fleiss JL (1981) Statistical methods for rates and proportions, 2nd edn. Wiley, New York

    Google Scholar 

  18. IBM Corp. Released 2011. IBM SPSS statistics for Windows, version 20.0. Armonk, IBM Corp

  19. Emil S, Nguyen T, Sills J et al (2004) Meconium obstruction in extremely low-birth-weight neonates: guidelines for diagnosis and management. J Pediatr Surg 39:731–737

    Article  PubMed  Google Scholar 

  20. Kane TD, Brown MF, Chen MK, Members of the APSA New Technology Committee (2009) Position paper on laparoscopic antireflux operations in infants and children for gastroesophageal reflux disease. American Pediatric Surgery Association. J Pediatr Surg 44:1034–1040

    Article  PubMed  Google Scholar 

  21. Bar-Joseph G, Galvis AG (1983) Perforation of the heart by central venous catheters in infants: guidelines to diagnosis and management. J Pediatr Surg 18:284–287

    Article  CAS  PubMed  Google Scholar 

  22. Stylianos S (2000) Evidence-based guidelines for resource utilization in children with isolated spleen or liver injury. APSA Trauma Committee. J Pediatr Surg 35:164–169

    Article  CAS  PubMed  Google Scholar 

  23. Okoye BO, Parikh DH, Buick RG et al (2000) Pyloric atresia: five new cases, a new association, and a review of the literature wi th guidelines. J Pediatr Surg 35:1241–1245

    Article  Google Scholar 

  24. Banieghbal B, Davies MR (2003) Guidelines for the successful treatment of lymphangioma with OK-432. Eur J Pediatr Surg 13:103–107

    Article  CAS  PubMed  Google Scholar 

  25. Greene AK, Liu AS, Mulliken JB et al (2011) Vascular anomalies in 5621 patients: guidelines for referral. J Pediatr Surg 46:1784–1789

    Article  PubMed  Google Scholar 

  26. Pena A, Bischoff A, Breech L et al (2010) Posterior cloaca—further experience and guidelines for the treatment of an unusual anorectal malformation. J Pediatr Surg 45:1234–1240

    Article  PubMed  Google Scholar 

  27. Trauma Committee of the Canadian Association of Pediatric Surgeons (2008) Canadian Association of Pediatric Surgeons’ position statement on the use of all-terrain vehicles by children and youth. J Pediatr Surg 43:938–939

    Article  Google Scholar 

  28. CAPS Ethics and Legal Committee (2008) The Canadian Association of Pediatric Surgeons’ position paper on the pediatric surgeon and blood-borne pathogens. J Pediatr Surg 43:936–937

    Article  Google Scholar 

  29. Oxman AD, Lavis JN, Lewin S et al (2009) SUPPORT Tools for evidence-informed health Policymaking (STP) 1: what is evidence-informed policymaking? Health Res Policy Syst 7:S1

    Article  PubMed Central  PubMed  Google Scholar 

  30. Eccles MP, Grimshaw JM, Shekelle P et al (2012) Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implement Sci 7:60

    Article  PubMed Central  PubMed  Google Scholar 

  31. Whitstock MT (2003) Seeking evidence from medical research consumers as part of the medical research process could improve the uptake of research evidence. J Eval Clin Pract 9:213–224

    Article  PubMed  Google Scholar 

  32. Boote J, Telford R, Cooper C (2002) Consumer involvement in health research: a review and research agenda. Health Policy 61:213–236

    Article  PubMed  Google Scholar 

  33. Shackley P, Ryan M (1995) Involving consumers in health care decision making. Health Care Anal 3:196–204

    Article  CAS  PubMed  Google Scholar 

  34. Lavis JN (2009) How can we support the use of systematic reviews in policymaking? PLoS Med 6:e1000141

    Article  PubMed Central  PubMed  Google Scholar 

  35. Woolf S, Schunemann HJ, Eccles MP et al (2012) Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implement Sci 7:61

    Article  PubMed Central  PubMed  Google Scholar 

  36. Cabana MD, Rand CS, Powe NR et al (1999) Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 282:1458–1465

    Article  CAS  PubMed  Google Scholar 

  37. Baker R, Camosso-Stefinovic J, Gillies C et al (2005) Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Update of Cochrane Database Syst Rev 3:CD005470

    Google Scholar 

  38. Pannucci CJ, Wilkins WG (2010) Identifying and avoiding bias in research. Plast Reconstr Surg 126:619–625

    Article  CAS  PubMed Central  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Molly Barber for her assistance with editing the manuscript.

Conflict of interest

None.

Author information

Authors and Affiliations

  1. Department of Pediatric Surgery, Alberta Children’s Hospital, 2888 Shaganappi Trail NW, Calgary, AB, T3B 6A8, Canada

    Anna C. Shawyer

  2. Division of General Surgery, Western University, London, ON, Canada

    Michael H. Livingston

  3. Department of Medicine, University at Buffalo, The State University of New York, Buffalo, NY, USA

    Veena Manja

  4. Departments of Oncology and Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, ON, Canada

    Melissa C. Brouwers

Authors
  1. Anna C. Shawyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael H. Livingston
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Veena Manja
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Melissa C. Brouwers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna C. Shawyer.

Box 1 Scaled domain scoring

Box 1 Scaled domain scoring

  • Obtained score = sum of all item scores for all appraisers in a single domain

  • Maximum possible score = 7 (strongly agree) × (number of items within domain) × 4 (appraisers)

  • Minimum possible score = 1 (strongly disagree) × (number of items within domain) × 4 (appraisers)

  • Scaled domain score = (obtained score − minimum possible score)/(maximum possible score − minimum possible score) × 100

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shawyer, A.C., Livingston, M.H., Manja, V. et al. The quality of guidelines in pediatric surgery: can we all AGREE?. Pediatr Surg Int 31, 61–68 (2015). https://doi.org/10.1007/s00383-014-3623-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00383-014-3623-6

Keywords

Search

Navigation