Participatory System Dynamics Modeling: Increasing Stakeholder Engagement and Precision to Improve Implementation Planning in Systems

  • Lindsey Zimmerman
  • David W. Lounsbury
  • Craig S. Rosen
  • Rachel Kimerling
  • Jodie A. Trafton
  • Steven E. Lindley
Original Paper


Implementation planning typically incorporates stakeholder input. Quality improvement efforts provide data-based feedback regarding progress. Participatory system dynamics modeling (PSD) triangulates stakeholder expertise, data and simulation of implementation plans prior to attempting change. Frontline staff in one VA outpatient mental health system used PSD to examine policy and procedural “mechanisms” they believe underlie local capacity to implement evidence-based psychotherapies (EBPs) for PTSD and depression. We piloted the PSD process, simulating implementation plans to improve EBP reach. Findings indicate PSD is a feasible, useful strategy for building stakeholder consensus, and may save time and effort as compared to trial-and-error EBP implementation planning.


Implementation science Mental health Evidence-based practices System dynamics Participatory stakeholder engagement 



This project is a collaboration among local and national mental health stakeholders in the Veterans Health Administration (VA). We would like to acknowledge the hard work and dedication of the mental health staff in outpatient services who are working to provide Veterans high-quality care. Staff are actively engaged in this process and are committed to improving the timing and reach of evidence-based mental health care. We would also like to acknowledge the contributions of the Veterans Advisory Partnership for Operations and Research (VAPOR). VAPOR is an advisory council of Veterans with lived experience in recovery as VA mental health patients who now work as patient advocates and health care navigators. We are grateful for their expertise and ongoing partnership in this improvement effort. We acknowledge project support contributions from McKenzie Javorka, B.A., Cora Bernard, M.S. and Dan Wang, Ph.D. Finally, we acknowledge the expertise of two national program offices, the Program Evaluation Resource Center (PERC) and the National Center for PTSD, Dissemination and Training Division. The views and opinions of authors expressed in this manuscript do not necessarily state or reflect those of the United States Government or the Department of Veterans Affairs.


This pilot study was funded with in-kind contributions from the National Center for PTSD, Department of Veterans Affairs.

Compliance with Ethical Standards

Conflict of Interest

All the authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.


  1. Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9(1), 45. doi: 10.1007/s10488-010-0327-7 PubMedPubMedCentralCrossRefGoogle Scholar
  2. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public services sectors. Administrative Policy Mental Health Services Research, 38, 4–23. doi: 10.1186/1748-5908-9-45 CrossRefGoogle Scholar
  3. Bendoly, E. (2014). System dynamics understanding in projects: Information sharing, psychological safety, and performance effects. Production and Operations Management, 23(8), 1352–1369. doi: 10.1111/poms.12024 CrossRefGoogle Scholar
  4. Caro, J. J., Briggs, A. H., Siebert, U., & Kuntz, K. M. (2012). Modeling good research practices—overview: A report of the ISPOR-SMDM modeling good research practices task force-1. Value in Health, 15, 796–803. doi: 10.1016/j.jval.2012.06.012 PubMedCrossRefGoogle Scholar
  5. Centers for Medicare & Medicaid Services Alliance to Modernize Healthcare (CAMH). (2015). Independent Assessment of the Health Care Delivery Systems and Management Processes of the Department of Veterans Affairs (Volume 1: Integrated Report). Retrieved October, 2015 from
  6. Chambers, D., Glasgow, R., & Stange, K. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8, 117. doi: 10.1186/1748-5908-8-117 PubMedPubMedCentralCrossRefGoogle Scholar
  7. Chard, K. M., Ricksecker, E. G., Healy, E. T., Karlin, B. E., & Resick, P. A. (2012). Dissemination and experience with cognitive processing therapy. Journal of Rehabilitation Research and Development, 49, 667–678. doi: 10.1682/JRRD.2011.10.0198 PubMedCrossRefGoogle Scholar
  8. Cook, J. M., O’Donnell, C., Dinnen, S., Bernardy, N., Rosenheck, R., & Hoff, R. (2013). A formative evaluation of two evidence-based psychotherapies for PTSD in VA residential treatment programs. Journal of Traumatic Stress, 26, 56–63. doi: 10.1002/jts.21769 PubMedPubMedCentralCrossRefGoogle Scholar
  9. Damschroder, L., Aron, D., Keith, R., Kirsh, S., Alexander, J., & Lowery, J. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. doi: 10.1186/1748-5908-4-50 PubMedPubMedCentralCrossRefGoogle Scholar
  10. Damschroder, L. & Lowery, J. (2015). Efficient synthesis: Using qualitative comparative analysis (QCA) and the CFIR across diverse studies. Plenary presentation at the 2015 Society for Implementation Research Collaboration, Seattle, WA.Google Scholar
  11. DelliFraine, J. L., Langabeer, J. R., & Nembhard, I. M. (2010). Assessing the evidence of Six Sigma and Lean in the health care industry. Quality Management in Healthcare, 19(3), 211–225. doi: 10.1097/QMH.0b013e3181eb140e CrossRefGoogle Scholar
  12. Department of Veterans Affairs. (2008). Uniform mental health services in VA medical centers and clinics. In Veterans Health Administration (Ed.), VHA handbook 260.01. Washington, DC.Google Scholar
  13. Department of Veterans Affairs. (2014). VA mental health services: public report. Retrieved November, 2014 from
  14. Department of Veterans Affairs and Department of Defense. (2009a). The management of MDD working group. VA/DOD clinical practice guideline for management of major depressive disorder (MDD). Retrieved from
  15. Department of Veterans Affairs and Department of Defense. (2009b). VA/DoD clinical practice guideline for the management of substance use disorders. Retrieved from
  16. Department of Veterans Affairs and Department of Defense. (2010). VA/DoD clinical practice guideline for the management of post-traumatic stress. Retrieved from
  17. Derickson, R., Fishman, J., Osatuke, K., Teclaw, R., & Ramsel, D. (2015). Psychological safety and error reporting within veterans health administration hospitals. Journal of Patient Safety, 11(1), 60–66. doi: 10.1097/PTS.0000000000000082 PubMedCrossRefGoogle Scholar
  18. Doty, D. H., Glick, W. H., & Huber, G. P. (1993). Fit, equifinality, and organizational effectiveness: a test of two configurational theories. Academy of Management Journal, 36(6), 1196–1250. doi: 10.2307/256810 CrossRefGoogle Scholar
  19. Eftekhari, A., Ruzek, J. I., Crowley, J. J., Rosen, C. S., Greenbaum, M. A., & Karlin, B. E. (2013). Effectiveness of national implementation of prolonged exposure therapy in Veterans Affairs care. JAMA Psychiatry, 70, 49–955.CrossRefGoogle Scholar
  20. Elbogen, E. B., Wagner, H. R., Johnson, S. C., Kinneer, P., Kang, H., Vasterling, J. J., et al. (2013). Are Iraq and Afghanistan veterans using mental health services? New data from a national random-sample survey. Psychiatric Services, 64, 134–141. doi: 10.1176/ PubMedPubMedCentralCrossRefGoogle Scholar
  21. Finley, E. P., Garcia, H. A., Ketchum, N. S., McGeary, D. D., McGeary, C. A., Stirman, S. W., et al. (2015). Utilization of evidence-based psychotherapies in Veterans Affairs posttraumatic stress disorder outpatient clinics. Psychological services, 12(1), 73. doi: 10.1037/ser0000014 PubMedCrossRefGoogle Scholar
  22. Flaspohler, P., Stillman, L., Duffy, J. L., Wandersman, A., & Maras, M. (2008). Unpacking capacity: The intersection of research to practice and community centered models. American Journal of Community Psychology, 41(3–4), 182–196. doi: 10.1007/s10464-008-9162-3 PubMedCrossRefGoogle Scholar
  23. Forrester, J. W. (1985). The model versus a modeling process. System Dynamics Review, 1, 133–134. doi: 10.1002/sdr.4260010112 CrossRefGoogle Scholar
  24. Forrester, J. W. (1961). Industrial Dynamics. Cambridge: MIT Press.Google Scholar
  25. Garcia, H. A., McGeary, C. A., Finley, E. P., Ketchum, N. S., McGeary, D. D., & Peterson, A. L. (2015). Burnout among psychiatrists in the Veterans Health Administration. Burnout Research, 2(4), 108–114. doi: 10.1016/j.burn.2015.10.001 CrossRefGoogle Scholar
  26. Glasgow, R. E., Vinson, C., Chambers, D., Khoury, M. J., Kaplan, R. M., & Hunter, C. (2012). National Institutes of Health approaches to dissemination and implementation science: Current and future directions. American Journal of Public Health, 102(7), 1274–1281. doi: 10.2105/AJPH.2012.300755 PubMedPubMedCentralCrossRefGoogle Scholar
  27. Glasser, S., Ellis, W., Chin, J., Glazner, C. & Kane, V. (2014). A model for eliminating veteran homelessness in the USA. Presentation at the 32nd International Conference of the System Dynamics Society, Delft, The Netherlands.Google Scholar
  28. Harris, A. H., Humphreys, K., Bowe, T., Kivlahan, D. R., & Finney, J. W. (2009). Measuring the quality of substance use disorder treatment: Evaluating the validity of the Department of Veterans Affairs continuity of care performance measure. Journal of Substance Abuse Treatment, 36(3), 294–305. doi: 10.1016/j.jsat.2008.05.011 PubMedCrossRefGoogle Scholar
  29. Hermes, E. D., Hoff, R., & Rosenheck, R. A. (2014). Sources of the increasing number of Vietnam era veterans with a diagnosis of PTSD using VHA services. Psychiatric Services, 65(6), 830–832. doi: 10.1176/ PubMedCrossRefGoogle Scholar
  30. Homer, J., Jones, A., Seville, D., Essien, J., Milstein, B., & Murphy, D. (2004). The CDC’s diabetes systems modeling project: developing a new tool for chronic disease prevention and control. In 22nd International Conference of the System Dynamics Society (pp. 25–29). Retrieved from
  31. Hovmand, P. (2014). Community-based system dynamics modeling. New York: Springer.CrossRefGoogle Scholar
  32. Karlin, B. E., Brown, G. K., Trockel, M., Cunning, D., Zeiss, A. M., & Taylor, C. B. (2012). National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs health care system: therapist and patient-level outcomes. Journal of Consulting and Clinical Psychology, 80, 707–718. doi: 10.1037/a0029328 PubMedCrossRefGoogle Scholar
  33. Karlin, B., & Cross, G. (2014). From the laboratory to the therapy room: National dissemination and implementation of evidence-based psychotherapies in the U.S. Department of Veterans Affairs Health Care System. American Psychologist, 69(1), 19–33. doi: 10.1037/a0033888 PubMedCrossRefGoogle Scholar
  34. Köpetz, C., Faber, T., Fishbach, A., & Kruglanski, A. W. (2011). The multifinality constraints effect: How goal multiplicity narrows the means set to a focal end. Journal of Personality and Social Psychology, 100(5), 810–826. doi: 10.1037/a0022980 PubMedCrossRefGoogle Scholar
  35. Lich, K. H., Tian, Y., Beadles, C. A., Williams, L. S., Bravata, D. M., Cheng, E. M., et al. (2014). Strategic planning to reduce the burden of stroke among Veterans using simulation modeling to inform decision-making. Stroke, 45, 2078–2084. doi: 10.1161/STROKEAHA.114.004694 PubMedPubMedCentralCrossRefGoogle Scholar
  36. Luke, D. A., & Stamatakis, K. A. (2012). Systems science methods in public health: Dynamics, networks, and agents. Annual Review of Public Health, 33, 357–376. doi: 10.1146/annurev-publhealth-031210-101222 PubMedPubMedCentralCrossRefGoogle Scholar
  37. Lyon, A. R., Maras, M. A., Pate, C. M., Igusa, T., & Vander Stoep, A. (2015). Modeling the impact of school-based universal depression screening on additional service capacity needs: A system dynamics approach. Administration and Policy in Mental Health and Mental Health Services Research,. doi: 10.1007/s10488-015-0628-y Google Scholar
  38. Mabry, P. (2014). Simulation as a tool to inform health policy. Part 1: Introduction to health systems simulations for policy. Academy Health Webinar.Google Scholar
  39. Mabry, P. L., Milstein, B., Abraido-Lanza, A. F., Livingood, W. C., & Allegrante, J. P. (2013). Opening on window on systems science research in health promotion and public health. Health Education & Behavior, 40, 5S–8S. doi: 10.1177/1090198113503343 CrossRefGoogle Scholar
  40. Martinez-Moyana, I. J., & Richardson, G. P. (2013). Best practices in system dynamics modeling. System Dynamics Review, 29, 102–123. doi: 10.1002/sdr.1495 CrossRefGoogle Scholar
  41. Masnick, K., & McDonnell, G. (2010). A model linking clinical workforce skill mix planning to health and health care dynamics. Human Resources for Health, 8(1), 1. doi: 10.1186/1478-4491-8-11 CrossRefGoogle Scholar
  42. Mazzocato, P., Savage, C., Brommels, M., Aronsson, H., & Thor, J. (2010). Lean thinking in healthcare: A realist review of the literature. Quality and Safety in Health Care, 19(5), 376–382. doi: 10.1136/qshc.2009.037986 PubMedGoogle Scholar
  43. Meadows, D. H., & Robinson, J. M. (1985). The Electronic Oracle: Computer models and social decisions. NewYork: Wiley.Google Scholar
  44. Meadows, D. H., & Wright, D. (2008). Thinking in systems: A primer. White River Junction: Chelsea Green Publishing.Google Scholar
  45. Morecroft, J., & Sterman, J. (1994). Modeling for learning organizations. Portland, OR: Productivity Press.Google Scholar
  46. Mott, J., Hundt, N., Sansgiry, S., Mignogna, J., & Cully, J. (2014). Changes in psychotherapy utilization among veterans with depression, anxiety, and PTSD. Psychiatric Services, 65(1), 106–112. doi: 10.1176/ PubMedCrossRefGoogle Scholar
  47. Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., et al. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2), 123–157. doi: 10.1177/1077558711430690 PubMedCrossRefGoogle Scholar
  48. Proctor, E. K., Powell, B. J., Baumann, A. A., Hamilton, A. M., & Santens, R. L. (2012). Writing implementation research grant proposals: Ten key ingredients. Implement Science, 7(1), 96. doi: 10.1186/1748-5908-7-96 CrossRefGoogle Scholar
  49. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy In Mental Health, 38(2), 65–76. doi: 10.1007/s10488-010-0319-7 PubMedCrossRefGoogle Scholar
  50. Rahmandad, H., Repenning, N., & Sterman, J. (2009). Effects of feedback delay on learning. System Dynamics Review, 25, 309–338. doi: 10.1002/sdr.427 CrossRefGoogle Scholar
  51. Richardson, G. P., & Pugh, A. L. (1981). Introduction to system dynamics modeling with dynamo. Cambridge: MIT Press.Google Scholar
  52. Rogers, E. M. (2003). Diffusion of innovations. New York: Free Press.Google Scholar
  53. Ruzek, J., Karlin, B. E., & Zeiss, A. (2012). Implementation of evidence-based psychological treatments in the veterans health administration. In R. K. McHugh & D. H. Barlow (Eds.), Dissemination of evidence-based psychological treatments. New York: Oxford University Press.Google Scholar
  54. Rycroft-Malone, J. (2004). The PARIHS framework—a framework for guiding the implementation of evidence-based practice. Journal of Nursing Care Quality, 19(4), 297–304.PubMedCrossRefGoogle Scholar
  55. Rycroft-Malone, J., Kitson, A., Harvey, G., McCormack, B., Seers, K., Titchen, A., et al. (2002). Ingredients for change: Revisiting a conceptual framework. Quality and Safety in Health Care, 11(2), 174–180. doi: 10.1136/qhc.11.2.174 PubMedPubMedCentralCrossRefGoogle Scholar
  56. Scaccia, J. P., Cook, B. S., Lamont, A., Wandersman, A., Castellow, J., Katz, J., et al. (2015). A practical implementation science heuristic for organizational readiness: R = MC2. Journal of Community Psychology, 43(4), 484–501. doi: 10.1002/jcop.21698 PubMedPubMedCentralCrossRefGoogle Scholar
  57. Senge, P. M. (2006). The fifth discipline: The art and practice of the learning organization. Vancouver, BC: Broadway Business.Google Scholar
  58. Shiner, B., D’Avolio, L. W., Nguyen, T. M., Zayed, M. H., Young-Xu, Y., Desai, R. A., et al. (2013). Measuring use of evidence based psychotherapy for posttraumatic stress disorder. Administration and Policy in Mental Health and Mental Health Services Research, 40(4), 311–318. doi: 10.1007/s10488-012-0421-0 PubMedCrossRefGoogle Scholar
  59. Simon, H. (1991). Bounded rationality and organizational learning. Organization Science, 2(1), 125–134. doi: 10.1287/orsc.2.1.125 CrossRefGoogle Scholar
  60. Sterman, J. (1994). Learning in and about complex systems. System Dynamics Review, 10, 291–330. doi: 10.1002/sdr.4260100214 CrossRefGoogle Scholar
  61. Sterman, J. D. (2000). Business dynamics: Systems thinking and modeling for a complex world (Vol. 19). Boston: Irwin/McGraw-Hill.Google Scholar
  62. Sterman, J. D. (2006). Learning from evidence in a complex world. American Journal of Public Health, 6, 505–514. doi: 10.2105/AJPH.2005.066043 CrossRefGoogle Scholar
  63. Stetler, C., Legro, M., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., et al. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 23. doi: 10.1186/1748-5908-1-23 PubMedPubMedCentralCrossRefGoogle Scholar
  64. System Dynamics Review. Hoboken: John Wiley & Sons, Inc. Retrieved from
  65. U.S. Department of Housing and Urban Development. (2014). The 2014 Annual Homeless Assessment Report (AHAR) to congress: PART 1 point-in-time estimates of homelessness. Retrieved October 1, 2015 from
  66. U.S. Interagency Council on Homelessness. (2013). Ending homelessness among veterans: A report by the United States interagency council on homelessness. Retrieved October, 2015 from
  67. Vennix, J. (1996). Group model building: facilitating team learning using system dynamics. Chichester: Wiley.Google Scholar
  68. Ventana Systems, Inc. (2014). Vensim® Version 6.3.Google Scholar
  69. Vest, J. R., & Gamm, L. D. (2009). A critical review of the research literature on Six Sigma, Lean and StuderGroup’s Hardwiring Excellence in the United States: the need to demonstrate and communicate the effectiveness of transformation strategies in healthcare. Implementation Science, 4(1), 35. doi: 10.1186/1748-5908-4-35 PubMedPubMedCentralCrossRefGoogle Scholar
  70. Warren, K. (2004). Why has feedback systems thinking struggled to influence strategy and policy formulation? Suggestive evidence, explanations and solutions. Systems Research and Behavioral Science, 21(4), 331–347. doi: 10.1002/sres.651 CrossRefGoogle Scholar
  71. Warren, K. (2005). Improving strategic management with the fundamental principles of system dynamics. System Dynamics Review, 21(4), 329–350. doi: 10.1002/sdr.325 CrossRefGoogle Scholar
  72. Watts, B. V., Shiner, B., Zubkoff, L., Carpenter-Song, E., Ronconi, J. M., & Coldwell, C. M. (2014). Implementation of evidence-based psychotherapies for posttraumatic stress disorder in VA specialty clinics. Psychiatric Services, 65, 648–653. doi: 10.1176/ PubMedCrossRefGoogle Scholar
  73. Weiner, B. J. (2009). A theory of organizational readiness for change. Implementation Science, 4, 67. doi: 10.1186/1748-5908-4-67 PubMedPubMedCentralCrossRefGoogle Scholar
  74. Yanchus, N. J., Periard, D., Moore, S. C., Carle, A. C., & Osatuke, K. (2015). Predictors of job satisfaction and turnover intention in VHA mental health employees: A comparison between psychiatrists, psychologists, social workers, and mental health nurses. Human Service Organizations: Management, Leadership & Governance, 39(3), 219–244. doi: 10.1080/23303131.2015.1014953 Google Scholar

Copyright information

© Springer Science+Business Media New York (outside the USA) 2016

Authors and Affiliations

  • Lindsey Zimmerman
    • 1
    • 2
  • David W. Lounsbury
    • 3
  • Craig S. Rosen
    • 1
    • 4
  • Rachel Kimerling
    • 1
  • Jodie A. Trafton
    • 4
    • 5
  • Steven E. Lindley
    • 4
    • 6
  1. 1.National Center for PTSD, Dissemination and Training DivisionVeteran Affairs Palo Alto Health Care SystemMenlo ParkUSA
  2. 2.University of Washington School of MedicineSeattleUSA
  3. 3.Department of Epidemiology and Population Health, Department of Family and Social MedicineAlbert Einstein College of MedicineBronxUSA
  4. 4.Stanford University School of MedicinePalo AltoUSA
  5. 5.Program Evaluation Resource Center, Center for Innovation to ImplementationVeteran Affairs Palo Alto Health Care SystemMenlo ParkUSA
  6. 6.Veteran Affairs Palo Alto Health Care SystemMenlo ParkUSA

Personalised recommendations