Due to the gap in suicide-specific intervention training for mental health students and professionals, e-learning is one solution to improving provider skills in the Veterans Affairs (VA) health system. This study focused on the development and evaluation of an equivalent e-learning alternative to the Collaborative Assessment and Management of Suicidality (CAMS) in-person training approach at a Veteran Health Affairs medical center.
The study used a multicenter, randomized, cluster, and three group design. the development of e-CAMS was an iterative process and included pilot testing. Eligible and consenting mental health providers, who completed a CAMS pre-survey, were randomized. Provider satisfaction was assessed using the standard VA evaluation of training consisting of 20 items. Two post training focus groups, divided by learning conditions, were conducted to assess practice adoption using a protocol focused on experiences with training and delivery of CAMS.
A total of 215 providers in five sites were randomized to three conditions: 69 to e-learning, 70 to in-person, 76 to the control. The providers were primarily female, Caucasian, midlife providers. Based on frequency scores of satisfaction items, both learning groups rated the trainings positively. In focus groups representing divided by learning conditions, participants described positive reactions to CAMS training and similar individual and institutional barriers to full implementation of CAMS.
This is the first evaluation study of a suicide-specific e-learning training within the VA. The e-CAMS appears equivalent to the in-person CAMS in terms of provider satisfaction with training and practice adoption, consistent with other comparisons of training deliveries across specialty areas. Additional evaluation of provider confidence and adoption and patient outcomes is in progress. The e-CAMS has the potential to provide ongoing training for VA and military mental health providers and serve as a tutorial for psychiatrists in preparation for specialty boards.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Katz I. Lessons learned from mental health enhancement and suicide prevention activities in the Veterans Health Administration [editorial]. Am J Public Health. 2012;102(supp1):S14–6.
Department of Veterans Affairs, Health Services Research and Development Services. Strategies for suicide prevention in veterans. Washington DC: Department of Veterans Affairs; 2009.
U.S. Department of Health and Human Services (HHS) Office of the Surgeon General and National Action Alliance for Suicide Prevention. 2012 National Strategy for Suicide Prevention: Goals and Objectives for Action. Washington, DC: HHS, September 2012. http://www.surgeongeneral.gov/library/reports/national-strategy Accessed October 16, 2012. suicide-prevention/full-report.pdf.
Department of Veterans Affairs. VA issues new report on suicide data. Febuary 13, 2013. http://www.va.gov/opa/docs/Suicide-Data-Report-2012-final.pdf.
Knesper DJ, American Association of Suicidology, Suicide Prevention Resource Center. Continuity of care for suicide prevention and research: Suicide attempts and suicide deaths subsequent to discharge from the emergency department or psychiatry inpatient unit. Newton, MA: Education Development Center, Inc. 2010. http://www.sprc.org Accessed January 3, 2012.
National Action Alliance: Care and Critical Intervention Task Force. Suicide Care and Systems Framework. National Action Alliance; 2011. http://actionallianceforsuicideprevevention.org/ Accessed February 8, 2012.
Schmitz W, Allen M, Feldman B, et al. Preventing suicide through improved training in suicide risk assessment and care: an American Association of Suicidology Task Force Report addressing serious gaps in U.S. mental health training. Suicide Life Threat. 2012;42(3):292–304. doi:10.1111/j.1943-278X.2012.00090.x.
Means B, Toyama Y, Murphy R, et al. Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. US Center for Technology in Learning, Office of Planning, Evaluation, and Policy Development, U.S. Department of Education 2009. http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf Accessed May 30, 2012.
Lyons S, Tripp-Reimer T, Sorofman B, et al. VA QUERI Informatics Paper information technology for clinical guideline implementation: perceptions of multidisciplinary stakeholders. J Med Inform Assoc. 2005;12:64–71. doi:10.1197/jamia.M1495.
Jobes D. Managing suicidal risk: a collaborative approach. New York, NY: Guilford; 2006.
Magruder K, York J, Jobes D, et al. Patient and provider outcomes of e-learning training in CAMS. EDU 08-424. Health Services R&D, Department of Veterans Affairs. 8/1/09-9/30/13.
Matthieu M, Cross W, Batres A, et al. Evaluation of gatekeeper training for suicide prevention in veterans. Arch Suicide Res. 2008;12(2):148–54. doi:10.1080/13811110701857491.
Ellis T, Green K, Allen J, et al. Collaborative assessment and management of suicidality in an inpatient setting: results of a pilot study. Psychother. 2012;49(1):72–80.
Arkov K, Rosenbaum B, Christiansen L, et al. Treatment of suicidal patients: the collaborative assessment and management of suicidality. Ugeskr Laeger. 2008;170(3):149–53.
Comtois KA, Jobes DA, O’Connor S, et al. Collaborative assessment and management of suicidality (CAMS): feasibility trial for next-day appointment services. Dep Anx. 2011;28:963–72.
Jobes DA, Jacoby AM, Cimbolic P, et al. Assessment and treatment of suicidal clients in a university counseling center. J Couns Psychol. 1997;44:368–77.
Jobes DA, Wong SA, Conrad AK, et al. The collaborative assessment and management of suicidality versus treatment as usual: a retrospective study with suicidal outpatients. Suicide Life Threat. 2005;35(5):483–97.
Jobes DA, Kahn-Greene E, Greene JA, Goeke-Morey M, et al. Clinical improvements of suicidal outpatients: examining suicide status form responses as predictors and moderators. Arch Suicide Res. 2009;13(2):147–59.
Jobes D, Comtois K, Brenner L, et al. Clinical trial feasibility studies of the Collaborative Assessment and Management of Suicidality (CAMS). In: O’Connor R, Platt S, Gordon J, editors. International handbook of suicide prevention: research, policy & practice. West Sussex, UK: Wiley–Blackwell; 2011. p. 383–400.
Jobes DA. The collaborative assessment and management of suicidality (CAMS): an evolving evidence-based clinical approach to suicidal risk. Suic Life Threat Behav. 2012;42(6):640–53. doi:10.1111/j.1943-278X.2012.00119.x.
Bagley S, Munjas B, Shekelle P. A systematic review of suicide prevention programs for military or veterans. Suicide Life Threat. 2010;40(3):257–65.
Ghahramanlou-Halloway M, Cox D, Fritz D, et al. An evidence-informed guide for working with military women and veterans. Prof Psychol Res Pract. 2011;42(1):1–7. doi:10.1037/a0022322.
Shoenbaum M, Heissen R, Pearson J. Opportunities to improve interventions to reduce suicidality: civilian “best practices” for army consideration. Washington DC: Department Health Human Services, National Institute of Health, National Institute of Mental Health; 2009.
Joint Commission. Behavioral Health Care 2013 National Patient Safety Goals http://www.jointcommission.org/standards_information/npsgs.aspx Accessed June 15, 2013.
Cook DA, Levinson AJ, Garside S, et al. Internet-based learning in the health professions: a metaanalysis. JAMA. 2008;300(10):1181–96.
Fordis M, King J, Ballantyne C, et al. Comparison of the instructional efficacy of Internet-based CME with live interactive CME workshops: a randomized control trial. J Am Med Assoc. 2005;294:1043–51.
Smith F, Singleton A, Hilton S. General practitioner’s continuing education: a review of policies, strategies, and effectiveness, and their implications for the future. Br J Gen Pract. 1998;48(435):1689–95.
Wutoch R, Boren S, Balas A. e-Learning: a review of internet-based continuing medical education. J Contin Educ Health Prof. 2004;24:20–30.
Cook DA, Garside S, Levinson AJ, et al. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ. 2010;44:765–74.
Cook DA, Levinson AJ, Garside S, et al. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad Med. 2010;85(5):909–22. doi:10.1097/ACM.0b013e3181d6c319.
Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010; 10(12). http://www.biomedcentral.com1472-6920/10/12.
Williams R, Gatien G, Hagerty B. Design element alternatives for stress-management intervention websites. Nurs Outlook. 2011;59:286–91. doi:10.1016/j.outlook.2011.03.009. DOI:10.1016/j.outlook.2011.03.009#doilink.
Lintonen TP, Konu AI, Seedhouse D. Information technology in health promotion. Health Educ Res. 2008;23(3):560–6. doi:10.1093/her/cym00.
Murray E, Burns J, May C, et al. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci. 2011;6(1):6–16.
Hrastinski S. A study of asynchronous and synchronous e-learning methods discovered that each supports different purposes. Educ Quart. 2008;4:51–5.
Magruder K, York J, Marshall E, et al. Can web-based training improve management of suicidal patients? Presentation. 141st American Public Health Annual Meeting, Boston MA, November, 2013.
Gorrindo T, Baer L, Sanders KM, et al. Web-based simulation in psychiatry residency training: a pilot study. Acad Psychiatry. 2011;35:232–7. doi:10.1176/appi.ap.35.4.232.
Brouwers M, Makarski J, Durocher L, et al. E-learning interventions are comparable to user’s manual in a randomized trial of training strategies for the AGREE II. Imp Sci. 2011;681:3–10.
Maher K, Brower K, Mullan P, et al. Web-based didactic instruction on substance use disorders compares favorably with live-lecture format. Acad Psychiatry. 2013;37:165–70. doi:10.1176/appi.ap.12010001.
Pisani A, Cross W, Gould M. The assessment and management of suicide risk: state of workshop education. Suicide Life Threat Behav. 2011;41(3):255–76. doi:10.1111/j. 1943-278X.2011.00026.x.
Kok G, Schaalma H, Ruiter R, et al. Intervention mapping: protocol for applying health psychology theory to prevention programs. J Health Psychol. 2004;9:85–98. doi:10.1177/1359105304038379.
Bartholomew LK, Parcel G, Kok G. Intervention mapping: a process for developing theory and evidence-based health education programs. Health Educ Behav. 1998;25:545–83. doi:10.1177/109019819802500502.
Institute of Medicine of the National Academies. Health professions education: a bridge to quality. Washington, DC: The National Academies Press; 2003.
We want to acknowledge the actors and script authors: Keith Jennings, Catholic University of America; Delia Chariker, VA SATC Addictioin Therapist; David Koerner, VA Social Worker; Denise O'Neil, VA SATC Therapist; Jeff Walker, Veteran, Jonathan Coutas, Medical University of South Carolina Videographer, and Dan Gross, VA Psychologist.
This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration. Office of Research and Development. Health Services Research and Development. The study was approved by the joint University and VA IRB site and VA office of Research in the primary site and VA IRBs of the four satellite sites.
Rights and permissions
About this article
Cite this article
Marshall, E., York, J., Magruder, K. et al. Implementation of Online Suicide-Specific Training for VA Providers. Acad Psychiatry 38, 566–574 (2014). https://doi.org/10.1007/s40596-014-0039-5
- Intervention training
- Mental health students
- Mental health residents
- Veteran Health Affairs medical center