Participatory System Dynamics Modeling: Increasing Stakeholder Engagement and Precision to Improve Implementation Planning in Systems
Implementation planning typically incorporates stakeholder input. Quality improvement efforts provide data-based feedback regarding progress. Participatory system dynamics modeling (PSD) triangulates stakeholder expertise, data and simulation of implementation plans prior to attempting change. Frontline staff in one VA outpatient mental health system used PSD to examine policy and procedural “mechanisms” they believe underlie local capacity to implement evidence-based psychotherapies (EBPs) for PTSD and depression. We piloted the PSD process, simulating implementation plans to improve EBP reach. Findings indicate PSD is a feasible, useful strategy for building stakeholder consensus, and may save time and effort as compared to trial-and-error EBP implementation planning.
KeywordsImplementation science Mental health Evidence-based practices System dynamics Participatory stakeholder engagement
This project is a collaboration among local and national mental health stakeholders in the Veterans Health Administration (VA). We would like to acknowledge the hard work and dedication of the mental health staff in outpatient services who are working to provide Veterans high-quality care. Staff are actively engaged in this process and are committed to improving the timing and reach of evidence-based mental health care. We would also like to acknowledge the contributions of the Veterans Advisory Partnership for Operations and Research (VAPOR). VAPOR is an advisory council of Veterans with lived experience in recovery as VA mental health patients who now work as patient advocates and health care navigators. We are grateful for their expertise and ongoing partnership in this improvement effort. We acknowledge project support contributions from McKenzie Javorka, B.A., Cora Bernard, M.S. and Dan Wang, Ph.D. Finally, we acknowledge the expertise of two national program offices, the Program Evaluation Resource Center (PERC) and the National Center for PTSD, Dissemination and Training Division. The views and opinions of authors expressed in this manuscript do not necessarily state or reflect those of the United States Government or the Department of Veterans Affairs.
This pilot study was funded with in-kind contributions from the National Center for PTSD, Department of Veterans Affairs.
Compliance with Ethical Standards
Conflict of Interest
All the authors declare that they have no conflict of interest.
This article does not contain any studies with human participants performed by any of the authors.
- Centers for Medicare & Medicaid Services Alliance to Modernize Healthcare (CAMH). (2015). Independent Assessment of the Health Care Delivery Systems and Management Processes of the Department of Veterans Affairs (Volume 1: Integrated Report). Retrieved October, 2015 from http://www.va.gov/opa/choiceact/documents/assessments/Integrated_Report.pdf
- Cook, J. M., O’Donnell, C., Dinnen, S., Bernardy, N., Rosenheck, R., & Hoff, R. (2013). A formative evaluation of two evidence-based psychotherapies for PTSD in VA residential treatment programs. Journal of Traumatic Stress, 26, 56–63. doi: 10.1002/jts.21769 PubMedPubMedCentralCrossRefGoogle Scholar
- Damschroder, L., Aron, D., Keith, R., Kirsh, S., Alexander, J., & Lowery, J. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. doi: 10.1186/1748-5908-4-50 PubMedPubMedCentralCrossRefGoogle Scholar
- Damschroder, L. & Lowery, J. (2015). Efficient synthesis: Using qualitative comparative analysis (QCA) and the CFIR across diverse studies. Plenary presentation at the 2015 Society for Implementation Research Collaboration, Seattle, WA.Google Scholar
- Department of Veterans Affairs. (2008). Uniform mental health services in VA medical centers and clinics. In Veterans Health Administration (Ed.), VHA handbook 260.01. Washington, DC.Google Scholar
- Department of Veterans Affairs. (2014). VA mental health services: public report. Retrieved November, 2014 from http://www.mentalhealth.va.gov/docs/Mental_Health_Transparency_Report_11-24-14.pdf
- Department of Veterans Affairs and Department of Defense. (2009a). The management of MDD working group. VA/DOD clinical practice guideline for management of major depressive disorder (MDD). Retrieved from http://www.healthquality.va.gov/guidelines/MH/mdd/MDDFULL053013.pdf
- Department of Veterans Affairs and Department of Defense. (2009b). VA/DoD clinical practice guideline for the management of substance use disorders. Retrieved from http://www.healthquality.va.gov/guidelines/MH/sud/sud_full_601f.pdf
- Department of Veterans Affairs and Department of Defense. (2010). VA/DoD clinical practice guideline for the management of post-traumatic stress. Retrieved from http://www.healthquality.va.gov/guidelines/MH/ptsd/cpgPTSDFULL201011612c.pdf
- Elbogen, E. B., Wagner, H. R., Johnson, S. C., Kinneer, P., Kang, H., Vasterling, J. J., et al. (2013). Are Iraq and Afghanistan veterans using mental health services? New data from a national random-sample survey. Psychiatric Services, 64, 134–141. doi: 10.1176/appi.ps.004792011 PubMedPubMedCentralCrossRefGoogle Scholar
- Finley, E. P., Garcia, H. A., Ketchum, N. S., McGeary, D. D., McGeary, C. A., Stirman, S. W., et al. (2015). Utilization of evidence-based psychotherapies in Veterans Affairs posttraumatic stress disorder outpatient clinics. Psychological services, 12(1), 73. doi: 10.1037/ser0000014 PubMedCrossRefGoogle Scholar
- Forrester, J. W. (1961). Industrial Dynamics. Cambridge: MIT Press.Google Scholar
- Glasgow, R. E., Vinson, C., Chambers, D., Khoury, M. J., Kaplan, R. M., & Hunter, C. (2012). National Institutes of Health approaches to dissemination and implementation science: Current and future directions. American Journal of Public Health, 102(7), 1274–1281. doi: 10.2105/AJPH.2012.300755 PubMedPubMedCentralCrossRefGoogle Scholar
- Glasser, S., Ellis, W., Chin, J., Glazner, C. & Kane, V. (2014). A model for eliminating veteran homelessness in the USA. Presentation at the 32nd International Conference of the System Dynamics Society, Delft, The Netherlands.Google Scholar
- Harris, A. H., Humphreys, K., Bowe, T., Kivlahan, D. R., & Finney, J. W. (2009). Measuring the quality of substance use disorder treatment: Evaluating the validity of the Department of Veterans Affairs continuity of care performance measure. Journal of Substance Abuse Treatment, 36(3), 294–305. doi: 10.1016/j.jsat.2008.05.011 PubMedCrossRefGoogle Scholar
- Homer, J., Jones, A., Seville, D., Essien, J., Milstein, B., & Murphy, D. (2004). The CDC’s diabetes systems modeling project: developing a new tool for chronic disease prevention and control. In 22nd International Conference of the System Dynamics Society (pp. 25–29). Retrieved from http://www.donellameadows.org/wp-content/userfiles/Diabetes_SystemISDC041.pdf.
- Karlin, B. E., Brown, G. K., Trockel, M., Cunning, D., Zeiss, A. M., & Taylor, C. B. (2012). National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs health care system: therapist and patient-level outcomes. Journal of Consulting and Clinical Psychology, 80, 707–718. doi: 10.1037/a0029328 PubMedCrossRefGoogle Scholar
- Lich, K. H., Tian, Y., Beadles, C. A., Williams, L. S., Bravata, D. M., Cheng, E. M., et al. (2014). Strategic planning to reduce the burden of stroke among Veterans using simulation modeling to inform decision-making. Stroke, 45, 2078–2084. doi: 10.1161/STROKEAHA.114.004694 PubMedPubMedCentralCrossRefGoogle Scholar
- Lyon, A. R., Maras, M. A., Pate, C. M., Igusa, T., & Vander Stoep, A. (2015). Modeling the impact of school-based universal depression screening on additional service capacity needs: A system dynamics approach. Administration and Policy in Mental Health and Mental Health Services Research,. doi: 10.1007/s10488-015-0628-y Google Scholar
- Mabry, P. (2014). Simulation as a tool to inform health policy. Part 1: Introduction to health systems simulations for policy. Academy Health Webinar.Google Scholar
- Meadows, D. H., & Robinson, J. M. (1985). The Electronic Oracle: Computer models and social decisions. NewYork: Wiley.Google Scholar
- Meadows, D. H., & Wright, D. (2008). Thinking in systems: A primer. White River Junction: Chelsea Green Publishing.Google Scholar
- Morecroft, J., & Sterman, J. (1994). Modeling for learning organizations. Portland, OR: Productivity Press.Google Scholar
- Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., et al. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2), 123–157. doi: 10.1177/1077558711430690 PubMedCrossRefGoogle Scholar
- Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy In Mental Health, 38(2), 65–76. doi: 10.1007/s10488-010-0319-7 PubMedCrossRefGoogle Scholar
- Richardson, G. P., & Pugh, A. L. (1981). Introduction to system dynamics modeling with dynamo. Cambridge: MIT Press.Google Scholar
- Rogers, E. M. (2003). Diffusion of innovations. New York: Free Press.Google Scholar
- Ruzek, J., Karlin, B. E., & Zeiss, A. (2012). Implementation of evidence-based psychological treatments in the veterans health administration. In R. K. McHugh & D. H. Barlow (Eds.), Dissemination of evidence-based psychological treatments. New York: Oxford University Press.Google Scholar
- Senge, P. M. (2006). The fifth discipline: The art and practice of the learning organization. Vancouver, BC: Broadway Business.Google Scholar
- Shiner, B., D’Avolio, L. W., Nguyen, T. M., Zayed, M. H., Young-Xu, Y., Desai, R. A., et al. (2013). Measuring use of evidence based psychotherapy for posttraumatic stress disorder. Administration and Policy in Mental Health and Mental Health Services Research, 40(4), 311–318. doi: 10.1007/s10488-012-0421-0 PubMedCrossRefGoogle Scholar
- Sterman, J. D. (2000). Business dynamics: Systems thinking and modeling for a complex world (Vol. 19). Boston: Irwin/McGraw-Hill.Google Scholar
- Stetler, C., Legro, M., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., et al. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 23. doi: 10.1186/1748-5908-1-23 PubMedPubMedCentralCrossRefGoogle Scholar
- System Dynamics Review. Hoboken: John Wiley & Sons, Inc. Retrieved from http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1099-1727
- U.S. Department of Housing and Urban Development. (2014). The 2014 Annual Homeless Assessment Report (AHAR) to congress: PART 1 point-in-time estimates of homelessness. Retrieved October 1, 2015 from https://www.hudexchange.info/resources/documents/2014-AHAR-Part1.pdf
- U.S. Interagency Council on Homelessness. (2013). Ending homelessness among veterans: A report by the United States interagency council on homelessness. Retrieved October, 2015 from http://usich.gov/resources/uploads/asset_library/USICH_Ending_Homelessness_Among_Veterans_Rpt_February_2013_FINAL.pdf
- Vennix, J. (1996). Group model building: facilitating team learning using system dynamics. Chichester: Wiley.Google Scholar
- Ventana Systems, Inc. (2014). Vensim® Version 6.3.Google Scholar
- Vest, J. R., & Gamm, L. D. (2009). A critical review of the research literature on Six Sigma, Lean and StuderGroup’s Hardwiring Excellence in the United States: the need to demonstrate and communicate the effectiveness of transformation strategies in healthcare. Implementation Science, 4(1), 35. doi: 10.1186/1748-5908-4-35 PubMedPubMedCentralCrossRefGoogle Scholar
- Watts, B. V., Shiner, B., Zubkoff, L., Carpenter-Song, E., Ronconi, J. M., & Coldwell, C. M. (2014). Implementation of evidence-based psychotherapies for posttraumatic stress disorder in VA specialty clinics. Psychiatric Services, 65, 648–653. doi: 10.1176/appi.ps.201300176 PubMedCrossRefGoogle Scholar
- Yanchus, N. J., Periard, D., Moore, S. C., Carle, A. C., & Osatuke, K. (2015). Predictors of job satisfaction and turnover intention in VHA mental health employees: A comparison between psychiatrists, psychologists, social workers, and mental health nurses. Human Service Organizations: Management, Leadership & Governance, 39(3), 219–244. doi: 10.1080/23303131.2015.1014953 Google Scholar