A Quasi-Experimental Analysis of Lethal Means Assessment and Risk for Subsequent Suicide Attempts and Deaths

Abstract

Background

Counseling on access to lethal means is highly recommended for patients with suicide risk, but there are no formal evaluations of its impact in real-world settings.

Objective

Evaluate whether lethal means assessment reduces the likelihood of suicide attempt and death outcomes.

Design

Quasi-experimental design using an instrumental variable to overcome confounding due to unmeasured patient characteristics that could influence provider decisions to deliver lethal means assessment.

Setting

Kaiser Permanente Colorado, an integrated health system serving over 600,000 members, with comprehensive capture of all electronic health records, medical claims, and death information.

Participants

Adult patients who endorsed suicide ideation on the Patient Health Questionnaire-9 (PHQ-9) depression screener administered in behavioral health and primary care settings from 2010 to 2016.

Interventions

Provider documentation of lethal means assessment in the text of clinical notes, collected using a validated Natural Language Processing program.

Measurements

Main outcome was ICD-9 or ICD-10 codes for self-inflicted injury or suicide death within 180 days of index PHQ-9 event.

Results

We found 33% of patients with suicide ideation reported on the PHQ-9 received lethal means assessment in the 30 days following identification. Lethal means assessment reduced the risk of a suicide attempt or death within 180 days from 3.3 to 0.83% (p = .034, 95% CI = .069–.9).

Limitations

Unmeasured suicide prevention practices that co-occur with lethal means assessment may contribute to the effects observed.

Conclusions

Clinicians should expand the use of counseling on access to lethal means, along with co-occurring suicide prevention practices, to all patients who report suicide ideation.

INTRODUCTION

Suicide mortality rates in the USA increased from 10.5 to 14 deaths per 100,000 between 1999 and 2017.1 Suicide is now the fourth leading cause of death for people aged 35–54 and the second leading cause of death for people aged 10–34.1 Current strategies to reduce suicide focus on assessing and reducing suicidal intent. An adjunctive suicide prevention intervention is restricting access to common lethal means used for suicide. Suicidal intent is impulsive or not planned well in advance in 82% of attempts2, and thought to last from only 5 min to 1 h.2,3 The opportunity for a clinician to intervene effectively when suicide intent arises is dependent on the patient accessing care during times of crisis. However, by restricting means, there is a barrier to self-harm that is not reliant on voluntary access to care in a time of crisis.

Restricting access to means through policy initiatives has been shown to be effective in reducing suicide outcomes.4 Examples include regulation of pesticides commonly used for suicide in China, firearm take back programs in Israel, and barricades on bridges in the USA.5,6,7,8 Based on this evidence, several medical and mental health authorities9,10,11,12,13 highly recommended counseling to remove access to lethal means in patients with suicide risk, such as locking up medications and removing or securing firearms, but effectiveness has yet to be established. A 2016 report from the Joint Commission recommended that both mental health and general healthcare settings, such as primary care and emergency departments, counsel patients with suicide risk and their families to remove access to lethal means.9 Restricting access to firearms holds a great deal of promise because firearms are the deadliest lethal mean for suicide in the USA, accounting for 50% of suicide deaths.14 Unlike politically charged initiatives that involve regulatory barriers to access, particularly those involving firearms, interventions in the medical and behavioral health setting are voluntary, and reliant on patients, their family members, and/or friends to take the necessary steps to prevent access. While voluntary interventions may be less effective than mandated policies, a key advantage is the potential to apply these practices broadly in any medical or behavioral health setting, regardless of political climate.

An important distinction of our work is that any patient may be assessed for access, but only those who screen positive for access (e.g., have a firearm in the home) will subsequently receive counseling on restricting access. The present study examines exposure to assessment only (e.g., provider asks patient: “do you have access to a firearm?”). The desired outcome of lethal means counseling is to put time and space between a patient at risk for suicide and common lethal means for suicide such as firearms, medications, ropes, and knives. Clinicians will discuss options for removing or locking-up dangerous items temporarily until the suicidal crisis passes. Despite recommendations, a handful of studies have shown that assessment for access to lethal means is intermittently delivered by medical and behavioral health providers.15,16,17 Qualitative studies show that providers may not prioritize lethal means assessment of suicidal patients because they do not believe it is effective.17,18,19

Lethal means assessment is commonly included as part of a battery of suicide prevention practices. Therefore, the causal mechanism related to lethal means assessment may include counseling on restricting access to lethal means plus other co-occurring suicide prevention practices, such as identifying reasons for living and coping strategies. Practices that regularly perform lethal means assessment may also provide better overall treatment for patients with suicide risk.

The present study examined the impact of lethal means assessment on suicide attempt and death outcomes within a large integrated health system. Specifically, assessment for access to firearms was measured in the clinical notes of medical and mental health providers within a group of patients who reported suicide ideation. Prior analysis of this sample indicated that providers who asked about firearm access were more likely to ask about other lethal means, such as toxic medications, ropes, and sharps, which suggests that assessment for access to firearms may be an indicator for assessment of other lethal means.20

METHODS

Study Sample

The study sample included electronic health record (EHR) data for patients at risk for suicide who were adult members (≥ 18 years) of the Kaiser Permanente Colorado (KPCO) health system for at least 2 years from 2010 to 2016 to ensure adequate data capture of exposures and outcomes. KPCO is an integrated closed panel system that provides salary-based primary and medical specialty care, and outpatient mental health care services to approximately 600,000 members, with the majority residing in the Denver/Boulder metropolitan region. This study was approved by the Kaiser Permanente and Colorado Multiple Institutional Review Boards.

We examined patients who reported suicide ideation of “more than half the days” or “nearly every day” in the past 2 weeks in response to the suicide ideation question (item 9) of the PHQ-9 depression questionnaire. These answers convey a 3–4% absolute risk for suicide attempt in the following year.21 The PHQ-9 is routinely administered in the waiting area prior to behavioral health appointments as well as to patients treated for depression in the primary care setting. PHQ-9 information is electronically imported into a questionnaire in the EHR, which we extracted. Patients were required to have at least 1 year of membership with no more than a 2-month membership gap, prior to first PHQ-9, which we considered the index event visit, to ensure adequate capture of covariate risk factors. This also ensured the index event PHQ-9 was either an initial or new episode of suicide risk. Patients were required to have 1 year of membership with no more than a 1-month membership gap after the PHQ-9 to be considered eligible for adequate capture of outcomes. Additionally, there had to be at least one encounter of any type on the PHQ-9 date or within the month following, no prior evidence of positive suicide ideation on the PHQ-9 or previous suicide attempts, and no evidence of any long-term care, palliative or hospice care encounters at any time. Patients with Medicaid insurance were excluded because outpatient mental health care is carved out and treatment records were not accessible from community mental health centers for this study to determine exposure to lethal means assessment.

Measure Construction

Documentation of assessment for access to firearms was determined using a validated natural language processing (NLP) query that was applied to all medical records with extractable text, including emergency and inpatient encounters from local hospitals for 30 days following index events (PHQ-9 administration).20,22 Manual chart review to determine accuracy of the NLP query for identifying firearm assessment showed that it had 97% sensitivity, 87% specificity, 82% positive predictive value, and 98% negative predictive value.20 Examples of text used as evidence of firearm assessment include “No access to firearms” and “Spouse has firearms that are locked up in gun safe. Patient does not have access to the key.” Nearly all patients who received assessment for access to medications or other means in the validation work also received firearm assessment. Additionally, the NLP query accuracy was highest for firearms compared with other means; therefore, we used firearm access assessment as an indicator of exposure to any type of lethal means assessment. The results of the firearm NLP query were used to create a dichotomous variable indicating any lethal means assessment during the 30 days following index event.

Main Outcomes

The primary dependent variable was occurrence of a suicide attempt, determined by either External Cause of Injury code (E950-958), ICD-10 code, or suicide death, determined by ICD-10 cause of death code from 7 to 180 days following an index event date (outcomes that occurred within the first 7 days were excluded to avoid potential misclassification of exposures as outcomes). This measure is based on previously validated methods for identifying self-injurious behavior in the emergency setting23 and AHRQ recommended ICD-10 codes for self-inflicted injuries.24 Deaths were identified from medical records, the State of Colorado Vital Statistics, and the Social Security Administration’s Death Master File. No outcomes occurred prior to exposure to lethal means assessment. Most (67%) of lethal means assessments were detected on the index date and 80% occurred within the first 7 days. A proportional hazards survival model examined lethal means assessment on time to suicide attempt/death within 180 days.

Covariates

The primary data source for all covariates was KPCO’s Data Warehouse that contains EHR and claims data. Covariates included patient demographics (age, race, ethnicity, gender, insurance type) and the following, measured during the 2 years prior to the index date: behavioral health department (BH) utilization, mental health diagnoses (depression, bipolar, schizophrenia, anxiety, substance abuse), total dispenses of any psychotropic medication, and total fills of opioids. Psychotherapy and psychiatry visits (identified via procedure codes) in the month following index and the year of the index date were also included.

Statistical Analysis

Information from risk assessments for suicide, such as risk factors (e.g., family history of suicide, hopelessness, patient reports a suicide plan) or protective factors (e.g., social support), was not available via administrative codes. This unmeasured information could potentially influence decisions to provide lethal means assessment. Failure to control for this will lead to selection bias such that patients with the greatest unmeasured risk for suicide would be more likely to receive lethal means assessment. We sought to overcome this bias using an instrumental variables model.25,26,27,28,29,30 The rationale for, and implementation of, this model is explained in detail in the supplemental appendix.

SAS Studio was used to extract data from KPCO’s data warehouse, as well as develop an analytic dataset; Stata/IC version 15.1 was used to estimate the analytic models. We applied a two-stage residual inclusion approach, which is appropriate for dichotomous outcomes, to estimate the instrumental variable models.25 We used a proportional hazards survival specification to measure the effect of lethal means assessment on time to suicide within 180 days.

Role of Funding Source

The extraction of data to complete this research was supported by the Strategic Allocation of Resources Committee (Internal Pilot Grant Program) at Kaiser Permanente Colorado.

RESULTS

The 7447 adult patients who met inclusion criteria had a rate of lethal means assessment of 33% across all visits in the month following index PHQ-9 event. In unadjusted analyses reported in Table 1, many of the covariates were significantly different between those who did/did not receive assessment. Men, younger patients, and those with depression, bipolar, and anxiety disorders were more likely to receive lethal means assessment. Also, patients with more psychotherapy visits in the month following index date were more likely to receive assessment, which may represent greater opportunity for exposure and/or increased severity of suicidal symptoms. Most suicide outcomes were non-fatal suicide attempts which occurred within 6 months of index date (Table 2). We were underpowered to examine suicide deaths alone.

Table 1 Baseline Characteristics of Patients with Suicide Ideation (Unadjusted)
Table 2 Suicide Attempt and Death Outcomes with 180 Days

Instrumental Variable Regression

Our adjusted model predicted reduction in suicide risk from 3.3% for those who did not receive lethal means assessment to 0.83% for those who did, a 2.47% reduction in absolute risk (Table 3; full model results in Supplemental Table 2). This result was robust across different levels of patient demographic and diagnostic groups (Table 3). The instrument strength, an important indicator for the validity of the instrumental variable approach31, measured by the F-statistic in the first stage, was well above 10 (F(34, 7412) = 46.46, p < .001; Supplemental Table 1).

Table 3 Adjusted Risk of Suicide Outcomes at 180 Days in Patients with Suicide Ideation

Survival analysis indicated a hazard ratio for lethal means assessment of .23 (p = .046, 95% CI = .054–.977) at 180 days with a significant log-rank test between groups (chi2(1)=10.81, p = .001) which indicates that the adjusted slope of the survival function differs between assessment groups. The plot of − log(− log (S(t))) against log(t) showed parallel lines, indicating that the proportional hazard assumption is upheld.

Sensitivity Analyses

A logit regression model of suicide outcomes without the instrument shows that the co-efficient for lethal means assessment is > 1 and significant with adjusted risk ratio = 1.56 (p = .03, 95% CI = 1.045–2.33) (Supplemental Table 3). The change in the estimate when omitting the instrumental variable illustrates selection bias. Namely, in the absence of a universal lethal means assessment policy for patients at risk for suicide, patients with the highest risk for suicide are more likely to receive assessment.

DISCUSSION

We examined the impact of lethal means assessment in the mental health and medical settings on subsequent suicide behavior outcomes. We found lethal means assessment was effective among patients with suicide ideation, reducing risk for suicide outcomes from 3.3 to .83%. Prior studies have found that patients reporting suicide risk on the PHQ-9 have a 3–4% risk of a suicide attempt or death in the following year.21 Therefore, an intervention that reduces this risk to 0.83% is clinically meaningful, reflecting a 75% reduction in risk. Our main findings were robust across gender, race, and age groups indicating the broad applicability of lethal means assessment in patients with suicide risk (Table 3). The instrumental variable estimates are interpreted as local average treatment effects, reflecting the reduction in risk in individuals who received assessments due to the practice(s) they visited.

We hope that our findings encourage expansion of the practice of lethal means assessment beyond the relatively low levels measured in the current study of 33%. Providers are likely using clinical judgment for risk when delivering lethal means assessment within those who report suicide ideation because we observed that males and those with mood disorders are more likely to receive assessment. However, our findings indicate that all patients with suicide ideation would potentially benefit from lethal means interventions, not just the highest risk groups. Prior survey and qualitative studies indicate many providers doubt the effectiveness of counseling on access to lethal means to reduce suicide outcomes, which may explain the low rate of assessment we detected.17,18,19 Providers may be uncomfortable with firearm safety counseling or unfamiliar with options for secure firearm storage33,34, but there is evidence that these barriers can be improved through training.35,36 There are a handful of small studies with promising findings for patient adherence to lethal means safety recommendations in the adolescent and adult populations.37,38 We do not know the robustness of the lethal means counseling that was provided after the assessments. However, if less effective counseling was provided, our findings would be biased towards the null. It is also possible that merely asking questions about lethal means could lead to a “question-behavior” effect.39 In other words, just asking about a risk behavior can reduce performance of the risk behavior and increase performance of healthy behaviors.

There are several strengths and limitations to consider in our approach. Our validation work of the NLP method that detected exposure to firearm access assessment indicated we were more likely to over-identify lethal means assessment, which could misclassify some people as exposed. False positives would likely skew our results towards the null, and underestimate the impact of lethal means assessment on suicide outcomes. Additionally, we did not control for levels of cost-sharing (e.g., high deductible plans), which can impact utilization of mental health services. However, everyone in the sample had an index visit with opportunity to receive assessment.

A strength of our approach is overcoming selection bias using an instrumental variable design. The primary assumption of this approach is the exclusion restriction, which states that our instrumental variable (practice-based probability of lethal means assessment) cannot directly predict suicide outcomes, which can only be tested in theory. Possible unmeasured variables that would violate the exclusion restriction include suicide prevention practices that may co-occur with lethal means assessment and impact subsequent risk for suicide. Practices that tend to conduct lethal means assessments may also be more likely to provide other prevention practices.39 These may include identification of formal and informal supports, distractions, or coping strategies.40 Chart reviewers were not looking for these types of practices during the NLP development work; therefore, it is not possible to know whether such safety planning practices co-occurred with lethal means assessment. A recent study of safety planning that included lethal means assessment, plus follow-up contacts in the emergency setting, showed a reduction in suicidal behaviors, but no studies have evaluated the impact of individual safety planning practices in isolation on suicide outcomes.41

Expansion of NLP methods or providing structured coding tools within the EHR systems to accurately assess additional suicide prevention practices is required to more completely parse out the effects of lethal means assessment and co-occurring suicide prevention practices on suicide outcomes. Randomized designs would not be ethical; therefore, future studies will require a larger sample of patients to ensure adequate suicide death capture, better characterization of co-occurring suicide prevention and safety planning interventions, and/or comparative effectiveness designs with universal implementation of lethal means assessment versus usual care.

CONCLUSION

To our knowledge, this is the first study to examine the impact of lethal means assessment on suicide outcomes in mental health and medical settings. We found that lethal means assessment reduced absolute risk for suicide by 2.74% in patients who reported suicide ideation on a screening tool during visits within a large integrated health system, but that only 33% of patients received any lethal means assessment. The causal pathway for this finding may be a combination of lethal means assessment with co-occurring unmeasured suicide prevention practices.

Mental health and medical providers should consider increasing the use of lethal means assessment alongside other types of suicide prevention practices in patients who report suicide ideation. This is consistent with the recommendations of the Zero Suicide model for health and behavioral health care systems.10 “Counseling on Access to Lethal Means” or CALM training is freely available online.42 Along with provider training, implementing electronic medical record documentation templates and/or reminders would also help to standardize high-quality suicide prevention care that incorporates lethal means safety interventions. Despite limitations from a quasi-experimental design, our study clearly illustrates that lethal means assessment is an important quality indicator for suicide prevention that health and mental health practices should monitor and advance to all patients with suicide risk.

References

  1. 1.

    Hedegaard H, Curtin SC, Warner M. Suicide mortality in the United States, 1999–2017. NCHS Data Brief, no 330. Hyattsville, MD: National Center for Health Statistics. 2018.

  2. 2.

    Baca-Garcia E, Diaz-Sastre C, Basurte E. et al. A prospective study of the paradoxical relationship between impulsivity and lethality of suicide attempts. J Clin Psychiatry. 2001;62(7):560–564

    CAS  PubMed  Google Scholar 

  3. 3.

    Simon, TR, Swann, AC, Powell, KE. et al. Characteristics of impulsive suicide attempts and attempters. Suicide and Life-Threatening Behavior. 2001; 32, 49–59. doi:https://doi.org/10.1521/suli.32.1.5.49.24212

    CAS  PubMed  Google Scholar 

  4. 4.

    Lewiecki, ME, Miller, SA. Suicide, Guns, and Public Policy. Am J Public Health. 2013; 103(1): 27–31

    PubMed  PubMed Central  Google Scholar 

  5. 5.

    Mann J, Apter A, Bertolote J, et al. Suicide Prevention Strategies: A Systematic Review. JAMA. 2005;294(16):2064–2074. doi:https://doi.org/10.1001/jama.294.16.2064.

    CAS  PubMed  Google Scholar 

  6. 6.

    Van der Feltz-Cornelis, CM, Sarchiapone, M, Postuvan, V, et al. Best practice elements of multilevel suicide prevention strategies: a review of systematic reviews. Crisis: The Journal of Crisis Intervention and Suicide Prevention. 2011; 32(6), 319. https://doi.org/10.1027/0227-5910/a000109

    Google Scholar 

  7. 7.

    Yip, PSF, Caine, E, Yousuf, S. et al. Means restriction for suicide prevention. The Lancet. 2012;379.9834: 2393–2399.

    Google Scholar 

  8. 8.

    Lubin, G, Werbeloff, N, Halperin, D. et al. Decrease in suicide rates after a change of policy reducing access to firearms in adolescents: a naturalistic epidemiological study. Suicide Life Threat Behav. 2010; 40(5), 421–424.

    PubMed  Google Scholar 

  9. 9.

    Joint Commission Accredited Organizations. Detecting and treating suicide ideation in all settings. Sentinel Event Alert. Issue 56: February 24, 2016.

  10. 10.

    Suicide Prevention Resource Center (2015). Zero Suicide. http://zerosuicide.sprc.org/

  11. 11.

    Substance Abuse and Mental Health Services Administration (SAMHSA). Center for Substance Abuse Treatment. Substance Abuse and Suicide Prevention: Evidence and Implications—A White Paper. DHHS Pub. No. SMA-08-4352. Rockville, MD: Substance Abuse and Mental Health Services Administration. 2008.

  12. 12.

    American Psychiatric Association (APA). Practice guideline for the assessment and treatment of patients with suicidal behaviors. 2003; Washington, DC: Author.

  13. 13.

    US Department of Veteran Affairs. VA/DoD Clinical Practice Guidelines: Assessment and Management of Patients at Risk for Suicide. https://www.healthquality.va.gov/guidelines/mh/srb/ Accessed September 1, 2018

  14. 14.

    Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS). 2017. https://webappa.cdc.gov/sasweb/ncipc/leadcause.html. Accessed December 1, 2018

  15. 15.

    Barber, CW, Miller, MJ. Reducing a suicidal person’s access to lethal means of suicide: a research agenda. Am J Prev Med. 2014;47(3):S264-S272. https://doi.org/10.1016/j.amepre.2014.05.028

    PubMed  Google Scholar 

  16. 16.

    Britton PC, Bryan CJ, Valenstein M. Motivational interviewing for means restriction counseling with patients at risk for suicide. Cognitive and Behavioral Practice. 2016; 23(1):51–61. https://doi.org/10.1016/j.cbpra.2014.09.004

    Google Scholar 

  17. 17.

    Betz, ME, Miller, M, Barber, et al. Lethal means restriction for suicide prevention: beliefs and behaviors of emergency department providers. Depress Anxiety. 2013;30(10):1013–1020. https://doi.org/10.1002/da.22075

  18. 18.

    Grossman, J, Dontes, A, Kruesi, et al. Emergency nurses’ responses to a survey about means restriction: An adolescent suicide prevention strategy. J Am Psychiatr Nurses Assoc. 2003;9(77–85). https://doi.org/10.1016/S1078-3903(03)00112-5

  19. 19.

    Wintemute GJ, Betz ME, Ranney ML. Yes, You Can: Physicians, Patients, and Firearms. Ann Intern Med. 2016;165:205–213. doi: https://doi.org/10.7326/M15-2905

    PubMed  Google Scholar 

  20. 20.

    Boggs JM, Powers JD, Rohm LM, et al. Using Natural Language Processing to Enable Quality Improvement and Future Research for Patients at Risk of Suicide. J Patient Cent Res Rev. 2017;4(3):188–9.

    Google Scholar 

  21. 21.

    Simon GE, Rutter CM, Peterson D, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death?. Psychiatr Serv. 2013;64(12):1195–202. https://doi.org/10.1176/appi.ps.201200587

    PubMed  PubMed Central  Google Scholar 

  22. 22.

    Friedman, C, Hripcsak, G. Natural language processing and its future in medicine. Acad Med. 1999;74(8):890–5. http://www.columbia.edu/itc/hs/medinfo/g6080/misc/articles/friedman.pdf

    CAS  PubMed  Google Scholar 

  23. 23.

    Ahmedani, BK, Stewart, C, Simon, GE, et al. Racial/ethnic differences in healthcare visits made prior to suicide attempt across the United States. Med Care. 2015;53(5):430–435. https://dx.doi.org/10.1097%2FMLR.0000000000000335

    PubMed  PubMed Central  Google Scholar 

  24. 24.

    Agency for Health Research and Quality (AHRQ). ICD-10-CM/PCS Specification Version 6.0 Patient Safety Indicators Appendices K. Self-Inflicted Injury Codes. 2010. Accessed January 2, 2019. https://www.qualityindicators.ahrq.gov/Downloads/Modules/PSI/V60-ICD10/TechSpecs/PSI_Appendix_K.pdf

  25. 25.

    Terza, JV, Basu, A, & Rathouz, PJ. Two-stage residual inclusion estimation: addressing endogeneity in health econometric modeling. J Health Econ. 2008;27(3):531–543. https://doi.org/10.1016/j.jhealeco.2007.09.009

    PubMed  Google Scholar 

  26. 26.

    McClellan M, McNeil BJ, Newhouse JP. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA. 1994; 21;272(11):859–66.

  27. 27.

    Sasso, ATL. Use of instrumental variables methods in examining psychiatric readmissions. JAMA Psychiatry. 2017;74(8): 805–806.

    PubMed  Google Scholar 

  28. 28.

    Slade, EP, Jahn, DR, Regenold WT, et al. Association of electroconvulsive therapy with psychiatric readmissions in US hospitals. JAMA Psychiatry. 2017;74(8):798–804.

    PubMed  PubMed Central  Google Scholar 

  29. 29.

    Presley CJ, Tang D, Soulos PR, et al. Association of broad-based genomic sequencing with survival among patients with advanced non–small cell lung cancer in the community oncology setting. JAMA. 2018;320(5):469–77.

    CAS  PubMed  PubMed Central  Google Scholar 

  30. 30.

    Duggan, M. Do new prescription drugs pay for themselves? The case of second-generation antipsychotics. J Health Econ. 2005; 24(1):1–31.

    PubMed  Google Scholar 

  31. 31.

    Staiger, DO, Stock, JH. Instrumental variables regression with weak instruments. Econometrica. 1997;65:557–586.

    Google Scholar 

  32. 32.

    Shea, J. S. 1997. Instrument relevance in multivariate linear models: A simple measure. Rev Econ Stat. 79: 348–352.

    Google Scholar 

  33. 33.

    Betz, ME, Wintemute, GJ. Physician counseling on firearm safety: a new kind of cultural competence. JAMA. 2015;314(5):449–50. doi: https://doi.org/10.1001/jama.2015.7055.

    CAS  PubMed  PubMed Central  Google Scholar 

  34. 34.

    Brent, DA, Bridge, J. Firearms Availability and Suicide Evidence, Interventions, and Future Directions. Am Behav Sci. 2003;46(9):1192–1210. https://doi.org/10.1177%2F0002764202250662

    Google Scholar 

  35. 35.

    Slovak, K, Brewer, TW. Suicide and firearm means restriction: can training make a difference? Suicide Life Threat Behav. 2010;40(1):63–73 doi: https://doi.org/10.1521/suli.2010.40.1.63.

    PubMed  Google Scholar 

  36. 36.

    Kruesi, MJ, Grossman, J, Pennington, JM, et al. Suicide and violence prevention: Parent education in the emergency department. J Am Acad Child Adolesc Psychiatry. 1999;38:250–255. https://doi.org/10.1097/00004583-199903000-00010

    CAS  PubMed  Google Scholar 

  37. 37.

    McManus, BL, Kruesl, MJ, Dontes, et al. Child and adolescent suicide attempts: An opportunity for emergency departments to provide injury prevention education. Am J Emerg Med. 1997;15:357–360. doi:https://doi.org/10.1016/S0735-6757(97)90124-8

  38. 38.

    Sherman, ME, Burns, K, Ignelzi, J, et al. Firearms risk management in psychiatric care. Psychiatr Serv. 2001;52(8). https://doi.org/10.1176/appi.ps.52.8.1057

  39. 39.

    Wilding, S., Conner, M., Prestwich, A., Lawton, R., & Sheeran, P. (2019). Using the question-behavior effect to change multiple health behaviors: An exploratory randomized controlled trial. Journal of experimental social psychology, 81, 53–60.

    PubMed  PubMed Central  Google Scholar 

  40. 40.

    Stanley, B, Stoll, B, Whiteside, U, et al. Safety Planning and Means Reduction in Large Health Care Organizations. December 16, 2014. Accessed September 8, 2018: https://zerosuicide.sprc.org/webinar/safety-planning-and-means-reduction-large-health-care-organizations

  41. 41.

    Stanley B, Brown GK, Brenner LA, et al. Comparison of the Safety Planning Intervention With Follow-up vs Usual Care of Suicidal Patients Treated in the Emergency Department. JAMA Psychiatry. 2018;75(9):894–900. doi:https://doi.org/10.1001/jamapsychiatry.2018.1776

    PubMed  PubMed Central  Google Scholar 

  42. 42.

    Counseling on Access to Lethal Means. Suicide Prevention Resource Center. Accessed: October 23, 2019. https://www.sprc.org/resources-programs/calm-counseling-access-lethal-means-0

Download references

Acknowledgments

We are grateful to John David Powers, MS, for his assistance extracting data from the medical record at Kaiser Permanente Colorado for this study.

Funding

The extraction of data to complete this research was supported by an internal pilot grant at Kaiser Permanente Colorado.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jennifer M. Boggs.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 16 kb)

ESM 2

(DOCX 36 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Boggs, J.M., Beck, A., Ritzwoller, D.P. et al. A Quasi-Experimental Analysis of Lethal Means Assessment and Risk for Subsequent Suicide Attempts and Deaths. J GEN INTERN MED 35, 1709–1714 (2020). https://doi.org/10.1007/s11606-020-05641-4

Download citation