World Journal of Urology

, Volume 36, Issue 9, pp 1489–1494 | Cite as

Are clinical guidelines designed according to guidelines? Cross-sectional assessment of quality and transparency of clinical guidelines in urology

  • Roderick C. N. van den Bergh
  • Piet Ost
  • Christian Surcel
  • Massimo Valerio
  • Jurgen J. Fütterer
  • Giorgio Gandaglia
  • Prasanna Sooriakumaran
  • Derya Tilki
  • Igor Tsaur
  • Guillaume Ploussard
  • the European Association of Urology Working Party on Prostate Cancer (EAU-YAUWP)
Original Article



Guidelines and recommendations become increasingly important in clinical urologic practice. This study aims to inform clinicians using guidelines on how to evaluate the quality of the methodology and transparency of these documents.


The guidelines on management of castration-resistant prostate cancer of the American Urology Association, European Association of Urology, National Comprehensive Cancer Network, National Institute for Health and Care Excellence, European Society of Medical Oncology were reviewed using the AGREE-II tool (Appraisal of Guidelines for Research and Evaluation). We reported and compared the domain scores for the domains 1 scope and purpose, 2 stakeholder involvement, 3 rigor of development, 4 clarity of presentation, 5 applicability, and 6 editorial independence (100% indicates highest—best quality score).


The domains evaluated highest and with lowest variability were ‘editorial independence’ (92% {88—95%}) and ‘clarity of presentation’ (83% {72–90%}), while the domains with the lowest scores and most variability were ‘stakeholder involvement’ (56% {36–79%}) and ‘applicability’ (40% {30–63%}). Length and extent of detail of guidelines vary considerably, each with its own strengths and limitations and adapted to target users. Standard external review using AGREE criteria may be preferable. A formal search strategy was not performed. Findings may be outdated by guidelines’ updates.


Clinicians using practice guidelines need to be aware of the different domains of methodology and transparency used to assess the quality of guidelines contents and recommendations.

Patient summary

Urologists increasingly use guidelines for support in evidence-based recommendations in clinical practice. It is very important to know how to assess these documents. This study applies standard criteria to compare the design and background of different available guidelines on prostate cancer no longer responding to hormonal treatment.


Prostate cancer Guidelines AGREE tool Review 




Compliance with ethical standards



Supplementary material

345_2018_2278_MOESM1_ESM.docx (101 kb)
Supplementary material 1 (DOCX 101 kb)


  1. 1.
    Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J (1999) Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 318(7182):527–530CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Loeb S (2014) Guideline of guidelines: prostate cancer screening. BJU Int 114(3):323–325PubMedGoogle Scholar
  3. 3.
    Grilli R, Magrini N, Penna A, Mura G, Liberati A (2000) Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet 355(9198):103–106CrossRefPubMedGoogle Scholar
  4. 4.
    Shaneyfelt TM, Mayo-Smith MF, Rothwangl J (1999) Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA 281(20):1900–1905CrossRefPubMedGoogle Scholar
  5. 5.
    Burgers JS, Fervers B, Haugh M, Brouwers M, Browman G, Philip T, Cluzeau FA (2004) International assessment of the quality of clinical practice guidelines in oncology using the appraisal of guidelines and research and evaluation instrument. J Clin Oncol 22(10):2000–2007CrossRefPubMedGoogle Scholar
  6. 6.
    Brouwers M, Kho ME, Browman GP, Cluzeau F, Feder G, Fervers B, Hanna S, Makarski J (2010) On behalf of the AGREE next steps consortium. AGREE II: advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J 182:E839–E842CrossRefGoogle Scholar
  7. 7.
    Cookson MS, Roth BJ, Dahm P et al (2013) Castration-resistant prostate cancer: AUA Guideline. J Urol 190(2):429–438CrossRefPubMedGoogle Scholar
  8. 8.
    Cornford P, Bellmunt J, Bolla M et al (2017) EAU-ESTRO-SIOG guidelines on prostate cancer. Part II: treatment of relapsing, metastatic, and castration-resistant prostate cancer. Eur Urol 71(4):630–642CrossRefPubMedGoogle Scholar
  9. 9.
    NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®) (2016) Prostate Cancer. Version 1.2017 — December 16, 2016.
  10. 10.
    Clinical Guideline Full Guideline (2014) Prostate Cancer: diagnosis and treatment.
  11. 11.
    Parker C, Gillessen S, Heidenreich A, Horwich A (2015) ESMO guidelines committee. cancer of the prostate: ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol 26(Suppl 5):v69–v77CrossRefPubMedGoogle Scholar
  12. 12.
    Morgia G, Russo GI, Tubaro A et al (2016) Patterns of prescription and adherence to European association of urology guidelines on androgen deprivation therapy in prostate cancer: an Italian multicentre cross-sectional analysis from the choosing treatment for prostate cancer (CHOICE) study. BJU Int 117(6):867–873CrossRefPubMedGoogle Scholar
  13. 13.
    Falchook AD, Hendrix LH, Chen RC (2015) Guideline-discordant use of imaging during work-up of newly diagnosed prostate cancer. J Oncol Pract 11(2):e239–e246CrossRefPubMedGoogle Scholar
  14. 14.
    Messina C, Bignotti B, Tagliafico A et al (2017) A critical appraisal of the quality of adult musculoskeletal ultrasound guidelines using the AGREE II tool: an EuroAIM initiative. Insights Imaging 8(5):491–497CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Lei X, Liu F, Luo S et al (2017) Evaluation of guidelines regarding surgical treatment of breast cancer using the AGREE Instrument: a systematic review. BMJ Open 7(11):e014883CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Sharma R, Alla K, Pfeffer D et al (2017) An appraisal of practice guidelines for smoking cessation in people with severe mental illness. Aust N Z J Psychiatry 51(11):1106–1120CrossRefPubMedGoogle Scholar
  17. 17.
    Gupta M, McGauley J, Farkas A et al (2015) Clinical practice guidelines on prostate cancer: a critical appraisal. J Urol 193(4):1153–1158CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Roderick C. N. van den Bergh
    • 1
  • Piet Ost
    • 2
  • Christian Surcel
    • 3
  • Massimo Valerio
    • 4
  • Jurgen J. Fütterer
    • 5
  • Giorgio Gandaglia
    • 6
  • Prasanna Sooriakumaran
    • 7
  • Derya Tilki
    • 8
  • Igor Tsaur
    • 9
  • Guillaume Ploussard
    • 10
  • the European Association of Urology Working Party on Prostate Cancer (EAU-YAUWP)
  1. 1.Department of UrologyAntonius HospitalUtrechtThe Netherlands
  2. 2.Department of RadiotherapyGhent University HospitalGhentBelgium
  3. 3.Centre of Urological Surgery, Dialysis and Renal TransplantationFundeni Clinical InstituteBucharestRomania
  4. 4.Department of UrologyCentre Hospitalier Universitaire VaudoisLausanneSwitzerland
  5. 5.Department of Radiology and Nuclear MedicineRadboud University Medical CenterNijmegenThe Netherlands
  6. 6.Division of Oncology/Unit of UrologyURI, IRCCS Ospedale San RaffaeleMilanItaly
  7. 7.Nuffield Department of Surgical SciencesUniversity of OxfordOxfordUK
  8. 8.Department of UrologyMartini-Klinik Prostate Cancer Center and University Hospital, Hamburg-EppendorfHamburgGermany
  9. 9.Department of Urology and Pediatric Urology, University Medical CenterJohannes Gutenberg University MainzMainzGermany
  10. 10.Department of UrologyInstitut Universitaire du Cancer Toulouse - Oncopole, Saint Jean Languedoc HospitalToulouseFrance

Personalised recommendations