Academic Psychiatry

, Volume 38, Issue 5, pp 566–574 | Cite as

Implementation of Online Suicide-Specific Training for VA Providers

  • Elizabeth MarshallEmail author
  • Janet York
  • Kathryn Magruder
  • Derik Yeager
  • Rebecca Knapp
  • Mark L. De Santis
  • Louisa Burriss
  • Mary Mauldin
  • Stan Sulkowski
  • Charlene Pope
  • David A. Jobes
Empirical Report



Due to the gap in suicide-specific intervention training for mental health students and professionals, e-learning is one solution to improving provider skills in the Veterans Affairs (VA) health system. This study focused on the development and evaluation of an equivalent e-learning alternative to the Collaborative Assessment and Management of Suicidality (CAMS) in-person training approach at a Veteran Health Affairs medical center.


The study used a multicenter, randomized, cluster, and three group design. the development of e-CAMS was an iterative process and included pilot testing. Eligible and consenting mental health providers, who completed a CAMS pre-survey, were randomized. Provider satisfaction was assessed using the standard VA evaluation of training consisting of 20 items. Two post training focus groups, divided by learning conditions, were conducted to assess practice adoption using a protocol focused on experiences with training and delivery of CAMS.


A total of 215 providers in five sites were randomized to three conditions: 69 to e-learning, 70 to in-person, 76 to the control. The providers were primarily female, Caucasian, midlife providers. Based on frequency scores of satisfaction items, both learning groups rated the trainings positively. In focus groups representing divided by learning conditions, participants described positive reactions to CAMS training and similar individual and institutional barriers to full implementation of CAMS.


This is the first evaluation study of a suicide-specific e-learning training within the VA. The e-CAMS appears equivalent to the in-person CAMS in terms of provider satisfaction with training and practice adoption, consistent with other comparisons of training deliveries across specialty areas. Additional evaluation of provider confidence and adoption and patient outcomes is in progress. The e-CAMS has the potential to provide ongoing training for VA and military mental health providers and serve as a tutorial for psychiatrists in preparation for specialty boards.


Suicide Intervention training Mental health students Mental health residents Veteran Health Affairs medical center 



We want to acknowledge the actors and script authors: Keith Jennings, Catholic University of America; Delia Chariker, VA SATC Addictioin Therapist; David Koerner, VA Social Worker; Denise O'Neil, VA SATC Therapist; Jeff Walker, Veteran, Jonathan Coutas, Medical University of South Carolina Videographer, and Dan Gross, VA Psychologist.


This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration. Office of Research and Development. Health Services Research and Development. The study was approved by the joint University and VA IRB site and VA office of Research in the primary site and VA IRBs of the four satellite sites.


  1. 1.
    Katz I. Lessons learned from mental health enhancement and suicide prevention activities in the Veterans Health Administration [editorial]. Am J Public Health. 2012;102(supp1):S14–6.PubMedCrossRefPubMedCentralGoogle Scholar
  2. 2.
    Department of Veterans Affairs, Health Services Research and Development Services. Strategies for suicide prevention in veterans. Washington DC: Department of Veterans Affairs; 2009.Google Scholar
  3. 3.
    U.S. Department of Health and Human Services (HHS) Office of the Surgeon General and National Action Alliance for Suicide Prevention. 2012 National Strategy for Suicide Prevention: Goals and Objectives for Action. Washington, DC: HHS, September 2012. Accessed October 16, 2012. suicide-prevention/full-report.pdf.
  4. 4.
    Department of Veterans Affairs. VA issues new report on suicide data. Febuary 13, 2013.
  5. 5.
    Knesper DJ, American Association of Suicidology, Suicide Prevention Resource Center. Continuity of care for suicide prevention and research: Suicide attempts and suicide deaths subsequent to discharge from the emergency department or psychiatry inpatient unit. Newton, MA: Education Development Center, Inc. 2010. Accessed January 3, 2012.
  6. 6.
    National Action Alliance: Care and Critical Intervention Task Force. Suicide Care and Systems Framework. National Action Alliance; 2011. Accessed February 8, 2012.
  7. 7.
    Schmitz W, Allen M, Feldman B, et al. Preventing suicide through improved training in suicide risk assessment and care: an American Association of Suicidology Task Force Report addressing serious gaps in U.S. mental health training. Suicide Life Threat. 2012;42(3):292–304. doi: 10.1111/j.1943-278X.2012.00090.x.CrossRefGoogle Scholar
  8. 8.
    Means B, Toyama Y, Murphy R, et al. Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. US Center for Technology in Learning, Office of Planning, Evaluation, and Policy Development, U.S. Department of Education 2009. Accessed May 30, 2012.
  9. 9.
    Lyons S, Tripp-Reimer T, Sorofman B, et al. VA QUERI Informatics Paper information technology for clinical guideline implementation: perceptions of multidisciplinary stakeholders. J Med Inform Assoc. 2005;12:64–71. doi: 10.1197/jamia.M1495.CrossRefGoogle Scholar
  10. 10.
    Jobes D. Managing suicidal risk: a collaborative approach. New York, NY: Guilford; 2006.Google Scholar
  11. 11.
    Magruder K, York J, Jobes D, et al. Patient and provider outcomes of e-learning training in CAMS. EDU 08-424. Health Services R&D, Department of Veterans Affairs. 8/1/09-9/30/13.Google Scholar
  12. 12.
    Matthieu M, Cross W, Batres A, et al. Evaluation of gatekeeper training for suicide prevention in veterans. Arch Suicide Res. 2008;12(2):148–54. doi: 10.1080/13811110701857491.PubMedCrossRefGoogle Scholar
  13. 13.
    Ellis T, Green K, Allen J, et al. Collaborative assessment and management of suicidality in an inpatient setting: results of a pilot study. Psychother. 2012;49(1):72–80.CrossRefGoogle Scholar
  14. 14.
    Arkov K, Rosenbaum B, Christiansen L, et al. Treatment of suicidal patients: the collaborative assessment and management of suicidality. Ugeskr Laeger. 2008;170(3):149–53.PubMedGoogle Scholar
  15. 15.
    Comtois KA, Jobes DA, O’Connor S, et al. Collaborative assessment and management of suicidality (CAMS): feasibility trial for next-day appointment services. Dep Anx. 2011;28:963–72.CrossRefGoogle Scholar
  16. 16.
    Jobes DA, Jacoby AM, Cimbolic P, et al. Assessment and treatment of suicidal clients in a university counseling center. J Couns Psychol. 1997;44:368–77.CrossRefGoogle Scholar
  17. 17.
    Jobes DA, Wong SA, Conrad AK, et al. The collaborative assessment and management of suicidality versus treatment as usual: a retrospective study with suicidal outpatients. Suicide Life Threat. 2005;35(5):483–97.CrossRefGoogle Scholar
  18. 18.
    Jobes DA, Kahn-Greene E, Greene JA, Goeke-Morey M, et al. Clinical improvements of suicidal outpatients: examining suicide status form responses as predictors and moderators. Arch Suicide Res. 2009;13(2):147–59.PubMedCrossRefGoogle Scholar
  19. 19.
    Jobes D, Comtois K, Brenner L, et al. Clinical trial feasibility studies of the Collaborative Assessment and Management of Suicidality (CAMS). In: O’Connor R, Platt S, Gordon J, editors. International handbook of suicide prevention: research, policy & practice. West Sussex, UK: Wiley–Blackwell; 2011. p. 383–400.CrossRefGoogle Scholar
  20. 20.
    Jobes DA. The collaborative assessment and management of suicidality (CAMS): an evolving evidence-based clinical approach to suicidal risk. Suic Life Threat Behav. 2012;42(6):640–53. doi: 10.1111/j.1943-278X.2012.00119.x.CrossRefGoogle Scholar
  21. 21.
    Bagley S, Munjas B, Shekelle P. A systematic review of suicide prevention programs for military or veterans. Suicide Life Threat. 2010;40(3):257–65.CrossRefGoogle Scholar
  22. 22.
    Ghahramanlou-Halloway M, Cox D, Fritz D, et al. An evidence-informed guide for working with military women and veterans. Prof Psychol Res Pract. 2011;42(1):1–7. doi: 10.1037/a0022322.CrossRefGoogle Scholar
  23. 23.
    Shoenbaum M, Heissen R, Pearson J. Opportunities to improve interventions to reduce suicidality: civilian “best practices” for army consideration. Washington DC: Department Health Human Services, National Institute of Health, National Institute of Mental Health; 2009.Google Scholar
  24. 24.
    Joint Commission. Behavioral Health Care 2013 National Patient Safety Goals Accessed June 15, 2013.
  25. 25.
    Cook DA, Levinson AJ, Garside S, et al. Internet-based learning in the health professions: a metaanalysis. JAMA. 2008;300(10):1181–96.PubMedCrossRefGoogle Scholar
  26. 26.
    Fordis M, King J, Ballantyne C, et al. Comparison of the instructional efficacy of Internet-based CME with live interactive CME workshops: a randomized control trial. J Am Med Assoc. 2005;294:1043–51.CrossRefGoogle Scholar
  27. 27.
    Smith F, Singleton A, Hilton S. General practitioner’s continuing education: a review of policies, strategies, and effectiveness, and their implications for the future. Br J Gen Pract. 1998;48(435):1689–95.PubMedPubMedCentralGoogle Scholar
  28. 28.
    Wutoch R, Boren S, Balas A. e-Learning: a review of internet-based continuing medical education. J Contin Educ Health Prof. 2004;24:20–30.CrossRefGoogle Scholar
  29. 29.
    Cook DA, Garside S, Levinson AJ, et al. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ. 2010;44:765–74.PubMedCrossRefGoogle Scholar
  30. 30.
    Cook DA, Levinson AJ, Garside S, et al. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad Med. 2010;85(5):909–22. doi: 10.1097/ACM.0b013e3181d6c319.PubMedCrossRefGoogle Scholar
  31. 31.
    Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010; 10(12). http://www.biomedcentral.com1472-6920/10/12.
  32. 32.
    Williams R, Gatien G, Hagerty B. Design element alternatives for stress-management intervention websites. Nurs Outlook. 2011;59:286–91. doi: 10.1016/j.outlook.2011.03.009. DOI: 10.1016/j.outlook.2011.03.009#doilink.PubMedCrossRefGoogle Scholar
  33. 33.
    Lintonen TP, Konu AI, Seedhouse D. Information technology in health promotion. Health Educ Res. 2008;23(3):560–6. doi: 10.1093/her/cym00.PubMedCrossRefGoogle Scholar
  34. 34.
    Murray E, Burns J, May C, et al. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci. 2011;6(1):6–16.PubMedCrossRefPubMedCentralGoogle Scholar
  35. 35.
    Hrastinski S. A study of asynchronous and synchronous e-learning methods discovered that each supports different purposes. Educ Quart. 2008;4:51–5.Google Scholar
  36. 36.
    Magruder K, York J, Marshall E, et al. Can web-based training improve management of suicidal patients? Presentation. 141st American Public Health Annual Meeting, Boston MA, November, 2013.Google Scholar
  37. 37.
    Gorrindo T, Baer L, Sanders KM, et al. Web-based simulation in psychiatry residency training: a pilot study. Acad Psychiatry. 2011;35:232–7. doi: 10.1176/appi.ap.35.4.232.PubMedCrossRefGoogle Scholar
  38. 38.
    Brouwers M, Makarski J, Durocher L, et al. E-learning interventions are comparable to user’s manual in a randomized trial of training strategies for the AGREE II. Imp Sci. 2011;681:3–10.Google Scholar
  39. 39.
    Maher K, Brower K, Mullan P, et al. Web-based didactic instruction on substance use disorders compares favorably with live-lecture format. Acad Psychiatry. 2013;37:165–70. doi: 10.1176/appi.ap.12010001.PubMedCrossRefGoogle Scholar
  40. 40.
    Pisani A, Cross W, Gould M. The assessment and management of suicide risk: state of workshop education. Suicide Life Threat Behav. 2011;41(3):255–76. doi: 10.1111/j. 1943-278X.2011.00026.x.PubMedCrossRefPubMedCentralGoogle Scholar
  41. 41.
    Kok G, Schaalma H, Ruiter R, et al. Intervention mapping: protocol for applying health psychology theory to prevention programs. J Health Psychol. 2004;9:85–98. doi: 10.1177/1359105304038379.PubMedCrossRefGoogle Scholar
  42. 42.
    Bartholomew LK, Parcel G, Kok G. Intervention mapping: a process for developing theory and evidence-based health education programs. Health Educ Behav. 1998;25:545–83. doi: 10.1177/109019819802500502.PubMedCrossRefGoogle Scholar
  43. 43.
    Institute of Medicine of the National Academies. Health professions education: a bridge to quality. Washington, DC: The National Academies Press; 2003.Google Scholar

Copyright information

© Academic Psychiatry 2014

Authors and Affiliations

  • Elizabeth Marshall
    • 1
    Email author
  • Janet York
    • 1
  • Kathryn Magruder
    • 1
  • Derik Yeager
    • 1
  • Rebecca Knapp
    • 1
  • Mark L. De Santis
    • 1
  • Louisa Burriss
    • 1
  • Mary Mauldin
    • 1
  • Stan Sulkowski
    • 1
  • Charlene Pope
    • 1
  • David A. Jobes
    • 1
  1. 1.Medical University of South CarolinaCharlestonUSA

Personalised recommendations