Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A Quasi-Experimental Analysis of Lethal Means Assessment and Risk for Subsequent Suicide Attempts and Deaths



Counseling on access to lethal means is highly recommended for patients with suicide risk, but there are no formal evaluations of its impact in real-world settings.


Evaluate whether lethal means assessment reduces the likelihood of suicide attempt and death outcomes.


Quasi-experimental design using an instrumental variable to overcome confounding due to unmeasured patient characteristics that could influence provider decisions to deliver lethal means assessment.


Kaiser Permanente Colorado, an integrated health system serving over 600,000 members, with comprehensive capture of all electronic health records, medical claims, and death information.


Adult patients who endorsed suicide ideation on the Patient Health Questionnaire-9 (PHQ-9) depression screener administered in behavioral health and primary care settings from 2010 to 2016.


Provider documentation of lethal means assessment in the text of clinical notes, collected using a validated Natural Language Processing program.


Main outcome was ICD-9 or ICD-10 codes for self-inflicted injury or suicide death within 180 days of index PHQ-9 event.


We found 33% of patients with suicide ideation reported on the PHQ-9 received lethal means assessment in the 30 days following identification. Lethal means assessment reduced the risk of a suicide attempt or death within 180 days from 3.3 to 0.83% (p = .034, 95% CI = .069–.9).


Unmeasured suicide prevention practices that co-occur with lethal means assessment may contribute to the effects observed.


Clinicians should expand the use of counseling on access to lethal means, along with co-occurring suicide prevention practices, to all patients who report suicide ideation.

This is a preview of subscription content, log in to check access.


  1. 1.

    Hedegaard H, Curtin SC, Warner M. Suicide mortality in the United States, 1999–2017. NCHS Data Brief, no 330. Hyattsville, MD: National Center for Health Statistics. 2018.

  2. 2.

    Baca-Garcia E, Diaz-Sastre C, Basurte E. et al. A prospective study of the paradoxical relationship between impulsivity and lethality of suicide attempts. J Clin Psychiatry. 2001;62(7):560–564

  3. 3.

    Simon, TR, Swann, AC, Powell, KE. et al. Characteristics of impulsive suicide attempts and attempters. Suicide and Life-Threatening Behavior. 2001; 32, 49–59. doi:

  4. 4.

    Lewiecki, ME, Miller, SA. Suicide, Guns, and Public Policy. Am J Public Health. 2013; 103(1): 27–31

  5. 5.

    Mann J, Apter A, Bertolote J, et al. Suicide Prevention Strategies: A Systematic Review. JAMA. 2005;294(16):2064–2074. doi:

  6. 6.

    Van der Feltz-Cornelis, CM, Sarchiapone, M, Postuvan, V, et al. Best practice elements of multilevel suicide prevention strategies: a review of systematic reviews. Crisis: The Journal of Crisis Intervention and Suicide Prevention. 2011; 32(6), 319.

  7. 7.

    Yip, PSF, Caine, E, Yousuf, S. et al. Means restriction for suicide prevention. The Lancet. 2012;379.9834: 2393–2399.

  8. 8.

    Lubin, G, Werbeloff, N, Halperin, D. et al. Decrease in suicide rates after a change of policy reducing access to firearms in adolescents: a naturalistic epidemiological study. Suicide Life Threat Behav. 2010; 40(5), 421–424.

  9. 9.

    Joint Commission Accredited Organizations. Detecting and treating suicide ideation in all settings. Sentinel Event Alert. Issue 56: February 24, 2016.

  10. 10.

    Suicide Prevention Resource Center (2015). Zero Suicide.

  11. 11.

    Substance Abuse and Mental Health Services Administration (SAMHSA). Center for Substance Abuse Treatment. Substance Abuse and Suicide Prevention: Evidence and Implications—A White Paper. DHHS Pub. No. SMA-08-4352. Rockville, MD: Substance Abuse and Mental Health Services Administration. 2008.

  12. 12.

    American Psychiatric Association (APA). Practice guideline for the assessment and treatment of patients with suicidal behaviors. 2003; Washington, DC: Author.

  13. 13.

    US Department of Veteran Affairs. VA/DoD Clinical Practice Guidelines: Assessment and Management of Patients at Risk for Suicide. Accessed September 1, 2018

  14. 14.

    Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS). 2017. Accessed December 1, 2018

  15. 15.

    Barber, CW, Miller, MJ. Reducing a suicidal person’s access to lethal means of suicide: a research agenda. Am J Prev Med. 2014;47(3):S264-S272.

  16. 16.

    Britton PC, Bryan CJ, Valenstein M. Motivational interviewing for means restriction counseling with patients at risk for suicide. Cognitive and Behavioral Practice. 2016; 23(1):51–61.

  17. 17.

    Betz, ME, Miller, M, Barber, et al. Lethal means restriction for suicide prevention: beliefs and behaviors of emergency department providers. Depress Anxiety. 2013;30(10):1013–1020.

  18. 18.

    Grossman, J, Dontes, A, Kruesi, et al. Emergency nurses’ responses to a survey about means restriction: An adolescent suicide prevention strategy. J Am Psychiatr Nurses Assoc. 2003;9(77–85).

  19. 19.

    Wintemute GJ, Betz ME, Ranney ML. Yes, You Can: Physicians, Patients, and Firearms. Ann Intern Med. 2016;165:205–213. doi:

  20. 20.

    Boggs JM, Powers JD, Rohm LM, et al. Using Natural Language Processing to Enable Quality Improvement and Future Research for Patients at Risk of Suicide. J Patient Cent Res Rev. 2017;4(3):188–9.

  21. 21.

    Simon GE, Rutter CM, Peterson D, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death?. Psychiatr Serv. 2013;64(12):1195–202.

  22. 22.

    Friedman, C, Hripcsak, G. Natural language processing and its future in medicine. Acad Med. 1999;74(8):890–5.

  23. 23.

    Ahmedani, BK, Stewart, C, Simon, GE, et al. Racial/ethnic differences in healthcare visits made prior to suicide attempt across the United States. Med Care. 2015;53(5):430–435.

  24. 24.

    Agency for Health Research and Quality (AHRQ). ICD-10-CM/PCS Specification Version 6.0 Patient Safety Indicators Appendices K. Self-Inflicted Injury Codes. 2010. Accessed January 2, 2019.

  25. 25.

    Terza, JV, Basu, A, & Rathouz, PJ. Two-stage residual inclusion estimation: addressing endogeneity in health econometric modeling. J Health Econ. 2008;27(3):531–543.

  26. 26.

    McClellan M, McNeil BJ, Newhouse JP. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA. 1994; 21;272(11):859–66.

  27. 27.

    Sasso, ATL. Use of instrumental variables methods in examining psychiatric readmissions. JAMA Psychiatry. 2017;74(8): 805–806.

  28. 28.

    Slade, EP, Jahn, DR, Regenold WT, et al. Association of electroconvulsive therapy with psychiatric readmissions in US hospitals. JAMA Psychiatry. 2017;74(8):798–804.

  29. 29.

    Presley CJ, Tang D, Soulos PR, et al. Association of broad-based genomic sequencing with survival among patients with advanced non–small cell lung cancer in the community oncology setting. JAMA. 2018;320(5):469–77.

  30. 30.

    Duggan, M. Do new prescription drugs pay for themselves? The case of second-generation antipsychotics. J Health Econ. 2005; 24(1):1–31.

  31. 31.

    Staiger, DO, Stock, JH. Instrumental variables regression with weak instruments. Econometrica. 1997;65:557–586.

  32. 32.

    Shea, J. S. 1997. Instrument relevance in multivariate linear models: A simple measure. Rev Econ Stat. 79: 348–352.

  33. 33.

    Betz, ME, Wintemute, GJ. Physician counseling on firearm safety: a new kind of cultural competence. JAMA. 2015;314(5):449–50. doi:

  34. 34.

    Brent, DA, Bridge, J. Firearms Availability and Suicide Evidence, Interventions, and Future Directions. Am Behav Sci. 2003;46(9):1192–1210.

  35. 35.

    Slovak, K, Brewer, TW. Suicide and firearm means restriction: can training make a difference? Suicide Life Threat Behav. 2010;40(1):63–73 doi:

  36. 36.

    Kruesi, MJ, Grossman, J, Pennington, JM, et al. Suicide and violence prevention: Parent education in the emergency department. J Am Acad Child Adolesc Psychiatry. 1999;38:250–255.

  37. 37.

    McManus, BL, Kruesl, MJ, Dontes, et al. Child and adolescent suicide attempts: An opportunity for emergency departments to provide injury prevention education. Am J Emerg Med. 1997;15:357–360. doi:

  38. 38.

    Sherman, ME, Burns, K, Ignelzi, J, et al. Firearms risk management in psychiatric care. Psychiatr Serv. 2001;52(8).

  39. 39.

    Wilding, S., Conner, M., Prestwich, A., Lawton, R., & Sheeran, P. (2019). Using the question-behavior effect to change multiple health behaviors: An exploratory randomized controlled trial. Journal of experimental social psychology, 81, 53–60.

  40. 40.

    Stanley, B, Stoll, B, Whiteside, U, et al. Safety Planning and Means Reduction in Large Health Care Organizations. December 16, 2014. Accessed September 8, 2018:

  41. 41.

    Stanley B, Brown GK, Brenner LA, et al. Comparison of the Safety Planning Intervention With Follow-up vs Usual Care of Suicidal Patients Treated in the Emergency Department. JAMA Psychiatry. 2018;75(9):894–900. doi:

  42. 42.

    Counseling on Access to Lethal Means. Suicide Prevention Resource Center. Accessed: October 23, 2019.

Download references


We are grateful to John David Powers, MS, for his assistance extracting data from the medical record at Kaiser Permanente Colorado for this study.


The extraction of data to complete this research was supported by an internal pilot grant at Kaiser Permanente Colorado.

Author information

Correspondence to Jennifer M. Boggs.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material


(DOCX 16 kb)


(DOCX 36 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Boggs, J.M., Beck, A., Ritzwoller, D.P. et al. A Quasi-Experimental Analysis of Lethal Means Assessment and Risk for Subsequent Suicide Attempts and Deaths. J GEN INTERN MED (2020).

Download citation