Skip to main content

An International Needs Assessment Survey of Guideline Developers Demonstrates Variability in Resources and Challenges to Collaboration between Organizations

Abstract

Background

The development of rigorous, high-quality clinical guidelines increases the need for resources and skilled personnel within guideline-producing organizations. While collaboration between organizations provides a unique opportunity to pool resources and save time and effort, the collaboration presents its own unique challenges.

Objective

To assess the perceived needs and current challenges of guideline producers worldwide related to guideline development and collaboration efforts.

Design

Survey questions were developed by the Guidelines International Network and the US GRADE Network, pilot-tested among attendees of a guideline development workshop, and disseminated electronically using convenience and snowball sampling methods.

Participants

A total of 171 respondents representing 30 countries and more than 112 unique organizations were included in this analysis.

Main Measures

The survey included free-response, multiple-choice, and seven-point Likert-scale questions. Questions assessed respondents’ perceived value of guidelines, resource availability and needs, guideline development processes, and collaboration efforts of their organization.

Key Results

Time required to develop high-quality systematic reviews and guidelines was the most relevant need (median=7; IQR=5.5–7). In-house resources to conduct literature searches (median=4; IQR=3–6) and the resources to develop rigorous guidelines rapidly (median=4; IQR=2–5) were perceived as the least available resources. Difficulties reconciling differences in guideline methodology (median=6; IQR=4–7) and the time required to establish collaborative agreements (median=6; IQR=5–6) were the most relevant barriers to collaboration between organizations. Results also indicated a general need for improvement in conflict of interest (COI) disclosure policies.

Conclusion

The survey identified organizational challenges in supporting rigorous guideline development, including the time, resources, and personnel required. Connecting guideline developers to existing databases of high-quality systematic reviews and the use of freely available online platforms may facilitate guideline development. Guideline-producing organizations may also consider allocating resources to hiring or training personnel with expertise in systematic review methodologies or utilizing resources more effectively by establishing collaborations with other organizations.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Data Availability

The datasets generated during and/or analyzed during this study are available from the corresponding author on reasonable request.

References

  1. 1.

    Sultan S, Morgan RL, Murad MH, et al. A Theoretical Framework and Competency-Based Approach to Training in Guideline Development. J Gen Intern Med. 2020;35(2):561-7.

    Article  Google Scholar 

  2. 2.

    Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421-4.

    Article  Google Scholar 

  3. 3.

    National Academies of Sciences EaM. Standards for Developing Trustworthy Clinical Practice Guidelines 2011. Available from: http://www.nationalacademies.org/hmd/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust/Standards.aspx. Accessed 8 Dec 2020.

  4. 4.

    Schunemann HJ, Wiercioch W, Etxeandia I, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123-42.

    Article  Google Scholar 

  5. 5.

    Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-6.

    Article  Google Scholar 

  6. 6.

    Kavita Gupta CMS, and Darlene F. Russ-Eft. A Practical Guide to Needs Assessment, 2nd Ed. San Francisco: John Wiley & Sons, Inc.; 2007.

    Google Scholar 

  7. 7.

    Sun X, Ioannidis JP, Agoritsas T, Alba AC, Guyatt G. How to use a subgroup analysis: users' guide to the medical literature. JAMA. 2014;311(4):405-11.

    CAS  Article  Google Scholar 

  8. 8.

    Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

    Article  Google Scholar 

  9. 9.

    Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006-12.

    Article  Google Scholar 

  10. 10.

    Siedler MR, Lamadrid P, Humphries MN, et al. The quality of physical activity guidelines, but not the specificity of their recommendations, has improved over time: a systematic review and critical appraisal. Appl Physiol Nutr Metab. 2021;46(1):34-45.

    Article  Google Scholar 

  11. 11.

    Dixon C, Dixon PE, Sultan S, et al. Guideline developers in the United States were inconsistent in applying criteria for appropriate Grading of Recommendations, Assessment, Development and Evaluation use. J Clin Epidemiol. 2020;124:193-9.

    Article  Google Scholar 

  12. 12.

    Bancos I, Cheng T, Prokop LJ, Montori VM, Murad MH. Endocrine clinical practice guidelines in North America. A systematic assessment of quality. J Clin Epidemiol. 2012;65(5):520-5.

    Article  Google Scholar 

  13. 13.

    Martinez Garcia L, Pardo-Hernandez H, Sanabria AJ, Alonso-Coello P, Pregnancy Clinical Guideline Updating Working G. Continuous surveillance of a pregnancy clinical guideline: an early experience. Syst Rev. 2017;6(1):143.

    Article  Google Scholar 

  14. 14.

    Sanabria AJ, Pardo-Hernandez H, Ballesteros M, et al. The UpPriority tool was developed to guide the prioritization of clinical guideline questions for updating. J Clin Epidemiol. 2020;126:80-92.

    Article  Google Scholar 

  15. 15.

    Alonso-Coello P, Martinez Garcia L, Carrasco JM, et al. The updating of clinical practice guidelines: insights from an international survey. Implement Sci. 2011;6:107.

    Article  Google Scholar 

  16. 16.

    Schunemann HJ, Woodhead M, Anzueto A, et al. A vision statement on guideline development for respiratory disease: the example of COPD. Lancet. 2009;373(9665):774-9.

    Article  Google Scholar 

  17. 17.

    Fischer F, Lange K, Klose K, Greiner W, Kraemer A. Barriers and Strategies in Guideline Implementation-A Scoping Review. Healthcare (Basel). 2016;4(3):36.

  18. 18.

    Thomas J, Noel-Storr A, Marshall I, et al. Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol. 2017;91:31-7.

    Article  Google Scholar 

  19. 19.

    Noel-Storr A, Dooley G, Affengruber L, Gartlehner G. Citation screening using crowdsourcing and machine learning produced accurate results: Evaluation of Cochrane's modified Screen4Me service. J Clin Epidemiol. 2021;130:23-31.

    Article  Google Scholar 

  20. 20.

    Chen Y, Yang K, Marusic A, et al. A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement. Ann Intern Med. 2017;166(2):128-32.

    Article  Google Scholar 

  21. 21.

    Morgan RL, Florez I, Falavigna M, et al. Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations. Health Res Policy Syst. 2018;16(1):63.

    Article  Google Scholar 

  22. 22.

    Wang X, Chen Y, Akl EA, et al. The reporting checklist for public versions of guidelines: RIGHT-PVG. Implement Sci. 2021;16(1):10.

    Article  Google Scholar 

Download references

Acknowledgements

Members of USGN (in alphabetical order)

Philipp Dahm, MD, MHSc; Yngve Falck-Ytter, MD; Rebecca L. Morgan, PhD, MPH; M. Hassan Murad, MD; Reem A. Mustafa, MD, MPH, PhD; Madelin R. Siedler, MA, MS; Shahnaz Sultan, MD, MHSc

Members of Collaboration WG (in alphabetical order)

Murad Alam, MD, MSCI, MBA; Lisa A. Fatheree; Thomas S. D. Getchius; Pamela K. Ginex, EdD, RN, OCN; Priya Jakhmola; Emma McFarlane; Rebecca L. Morgan, PhD, MPH; Toju Ogunremi, MSc; Robyn L. Temple Smolkin, MBA, PhD, HCLD

Author information

Affiliations

Authors

Contributions

Project conceptualization: LF, TG, PG, RLM, TO, SS, RAM, MHM, YFY

Survey development: LF, PJ, TO, RTS, RLM, SS, RAM, MHM, MRS

Analysis: MRS, SS, RLM, BYK

Writing: SS, MRS, RAM, RLM

Reviewing and approval of final manuscript: All

Corresponding author

Correspondence to Shahnaz Sultan MD, MHSc.

Ethics declarations

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Shahnaz Sultan and Madelin R. Siedler are co-first authors.

Supplementary Information

ESM 1

(PDF 296 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sultan, S., Siedler, M.R., Morgan, R.L. et al. An International Needs Assessment Survey of Guideline Developers Demonstrates Variability in Resources and Challenges to Collaboration between Organizations. J GEN INTERN MED (2021). https://doi.org/10.1007/s11606-021-07112-w

Download citation

Key Words

  • guidelines
  • needs assessment
  • collaboration
  • guideline international network
  • GRADE