Abstract
This study examined suicide assessment validity by comparing methods of measuring current risk associated with past suicidal behaviors. Three independent samples (Ns = 359, 1007, and 713; aged 18–76 years) all included participants covering a broad spectrum of suicidality. Information theory, item response theory, general linear modeling, and linear regression modeling tested seven competing methods/models of assessing past suicidal behaviors in relation to current suicidality. In contrast to contemporary theories, ANOVA results showed suicide plans can indicate higher risk than suicide attempts when intent to die is higher. Contrary to popular practice, evidence demonstrated that defining risk by suicide ideation (yes/no), attempts or serious attempts (yes/no), are false dichotomies, were the least valid models tested, and failed to explain substantial explainable variance in suicidality/risk. A newly proposed model, differentiating behaviors with or without intent to die, was the most efficacious dichotomous method. However, as predicted, continuous variables were superior to dichotomous. The proposed suicidal barometer model (SBM) exhibited robust evidence as the best available model for evaluating suicidal behaviors in all samples (100 % probability), explaining 47–61 % of suicidality variance and provided incremental improvement in risk evaluations. Findings were consistent by sample, sex, age-group, ethnicity, and psychiatric history. This study, and related evidence, demonstrate that there is a clear and present need for updating measures, clinical training and core competencies, for valid assessment and risk formulation.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10862-016-9566-6/MediaObjects/10862_2016_9566_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10862-016-9566-6/MediaObjects/10862_2016_9566_Fig2_HTML.gif)
Similar content being viewed by others
References
Akaike, H. (1981). Likelihood of a model and information criteria. J Econ, 16(1), 3–14. doi:10.1016/0304–4076(81)90071–3.
Alwin, D. F. (2007). Margins of error: A study of reliability in survey measurement. Hoboken, NJ: Wiley.
Baca-Garcia, E., Perez-Rodriguez, M. M., Oquendo, M. A., Keyes, K. M., Hasin, D. S., Grant, B. F., & Blanco, C. (2011). Estimating risk for suicide attempt: Are we asking the right questions?: Passive suicidal ideation as a marker for suicidal behavior. J Affect Disord, 134(1–3), 327–332. doi:10.1016/j.jad.2011.06.026.
Beautrais, A. L. (2003). Subsequent mortality in medically serious suicide attempts: A 5 year follow-up. Aust N Z J Psychiatry, 37(5), 595–599.
Beck, R. W., Morris, J. B., & Beck, A. T. (1974). Cross-validation of the Suicidal Intent Scale. Psychol Rep, 34(2), 445–446.
Beck, A. T., Kovacs, M., & Weissman, A. (1979). Assessment of suicidal intention: The Scale for Suicide Ideation. J Consult Clin Psychol, 47(2), 343–352. doi:10.1037/0022-006X.47.2.343.
Beck, A. T., Brown, G. K., Steer, R. A., Dahlsgaard, K. K., & Grisham, J. R. (1999). Suicide ideation at its worst point: A predictor of eventual suicide in psychiatric outpatients. Suicide Life Threat Behav, 29(1), 1–9.
Bernert, R. A., Hom, M. A., & Roberts, L. W. (2014). A review of multidisciplinary clinical practice guidelines in suicide prevention: Toward an emerging standard in suicide risk assessment and management, training and practice. Acad Psychiatry, 38(5), 585–592. doi:10.1007/s40596–014–0180-1.
Bevans, K. B., Diamond, G., & Levy, S. (2012). Screening for adolescents' internalizing symptoms in primary care: Item response theory analysis of the behavior health screen depression, anxiety, and suicidal risk scales. J Dev Behav Pediatr, 33(4), 283–290. doi:10.1097/DBP.0b013e31824eaa9a.
Böhnke, J. R., & Croudace, T. J. (2015). Factors of psychological distress: Clinical value, measurement substance, and methodological artefacts. Soc Psychiatry Psychiatr Epidemiol, 50(4), 515–524. doi:10.1007/s00127–015–1022-5.
Bolton, C., Gooding, P., Kapur, N., Barrowclough, C., & Tarrier, N. (2007). Developing psychological perspectives of suicidal behaviour and risk in people with a diagnosis of schizophrenia: We know they kill themselves but do we understand why? Clin Psychol Rev, 27(4), 511–536. doi:10.1016/j.cpr.2006.12.001.
Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach (2nd ed.). New York, NY: Springer.
Centers for Disease Control and Prevention (2011). Suicidal thoughts and behaviors among adults aged ≥18 years — United States, 2008–2009. Morb Mortal Wkly Rep, 60(13), 1–22.
Chalker, S. A., Comtois, K. A., & Kerbrat, A. H. (2015). Impulsivity and suicidal behavior: How you define it matters. International Journal of Cognitive Therapy, 8(2), 172–192.
Chapman, E. N., Kaatz, A., & Carnes, M. (2013). Physicians and implicit bias: How doctors may unwittingly perpetuate health care disparities. J Gen Intern Med, 28(11), 1504–1510. doi:10.1007/s11606–013–2441-1.
Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis. Hillsdale, NJ: Erlbaum.
Coric, V., Stock, E. G., Pultz, J., Marcus, R., & Sheehan, D. V. (2009). Sheehan Suicidality Tracking Scale (Sheehan-STS): Preliminary results from a multicenter clinical trial in generalized anxiety disorder. Psychiatry, 6(1).
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 173–178.
De Beurs, D. P., De Vries, A. L. M., De Groot, M. H., De Keijser, J., & Kerkhof, A. J. F. M. (2014). Applying computer adaptive testing to optimize online assessment of suicidal behavior:a simulation study. Journal of Medical Internet Research, 16(9). doi:10.2196/jmir.3511.
de Beurs, E., Barendregt, M., Rogmans, B., Robbers, S., van Geffen, M., van Aggelen-Gerrits, M., & Houben, H. (2015). Denoting treatment outcome in child and adolescent psychiatry: A comparison of continuous and categorical outcomes. Eur Child Adolesc Psychiatry, 24(5), 553–563. doi:10.1007/s00787–014–0609-9.
DeVellis, R. F. (2012). Scale development: Theory and applications (Vol. 26, 3rd ed.). Los Angeles, CA: Sage.
Development Core Team, R. (2016). R: A language and environment for statistical computing (Version 3.2.4). Viena, Austria: R Foundation for Statistical Computing . Retrieved from http://www.R-project.org.
Dube, P., Kroenke, K., Bair, M. J., Theobald, D., & Williams, L. S. (2010). The P4 screener: Evaluation of a brief measure for assessing potential suicide risk in 2 randomized effectiveness trials of primary care and oncology patients. Primary Care Companion to the Journal of Clinical Psychiatry, 12(6). doi:10.4088/PCC.10m00978blu.
Farberow, N. L., & Shneidman, E. S. (Eds.) (1961). The cry for help. New York, NY: McGraw-Hill.
Fedorov, V., Mannino, F., & Zhang, R. (2009). Consequences of dichotomization. Pharm Stat, 8(1), 50–61. doi:10.1002/pst.331.
Fortune, S., Stewart, A., Yadav, V., & Hawton, K. (2007). Suicide in adolescents: Using life charts to understand the suicidal process. J Affect Disord, 100(1–3), 199–210.
Giner, L., Jaussent, I., Olie, E., Beziat, S., Guillaume, S., Baca-Garcia, E., et al. (2014). Violent and serious suicide attempters: One step closer to suicide? J Clin Psychiatry, 75(3), e191–e197. doi:10.4088/JCR13mO8524.
Gipson, P., & King, C. (2012). Health behavior theories and research: Implications for suicidal individuals' treatment linkage and adherence. Cognitive and Behavioral Practice, 19(2), 209–217. doi:10.1016/j.cbpra.2010.11.005.
Goldberg, J. F., Ernst, C. L., & Bird, S. (2007). Predicting hospitalization versus discharge of suicidal patients presenting to a psychiatric emergency service. Psychiatr Serv, 58(4), 561–565.
Groth-Marnat, G. (2009). Handbook of psychological assessment (5th ed.). Hoboken, NJ: John Wiley & Sons.
Gutierrez, P. M., Freedenthal, S., Wong, J. L., Osman, A., & Norizuki, T. (2012). Validation of the suicide resilience inventory-25 (SRI-25) in adolescent psychiatric inpatient samples. J Pers Assess, 94(1), 53–61. doi:10.1080/00223891.2011.608755.
Hamilton, M. (1967). Development of a rating scale for primary depressive illness. The British journal of social and clinical psychology, 6(4), 278–296.
Harris, K. M., McLean, J. P., Sheffield, J., & Jobes, D. (2010). The internal suicide debate hypothesis: Exploring the life versus death struggle. Suicide Life Threat Behav, 40(2), 181–192. doi:10.1521/suli.2010.40.2.181.
Harris, K. M., McLean, J. P., & Sheffield, J. (2014). Suicidal and online: How do online behaviors inform us of this high-risk population? Death Studies, 38(6), 387–394. doi:10.1080/07481187.2013.768313.
Harris, K. M., Syu, J.-J., Lello, O. D., Chew, Y. L. E., Willcox, C. H., & Ho, R. C. M. (2015). The ABC’s of suicide risk assessment: Applying a tripartite approach to individual evaluations. PLoS One, 10(6), e0127442. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0127442 doi:10.1371/journal.pone.0127442
Harriss, L., Hawton, K., & Zahl, D. (2005). Value of measuring suicidal intent in the assessment of people attending hospital following self-poisoning or self-injury. Br J Psychiatry, 186(1), 60–66.
Hjelmeland, H., Stiles, T. C., Bille-Brahe, U., Ostamo, A., Renberg, E. S., & Wasserman, D. (1998). Parasuicide: The value of suicidal intent and various motives as predictors of future suicidal behaviour. Archives of Suicide Research, 4(3), 209–225.
Horwitz, A. G., Czyz, E. K., & King, C. A. (2014). Predicting future suicide attempts among adolescent and emerging adult psychiatric emergency patients. J Clin Child Adolesc Psychol. doi:10.1080/15374416.2014.910789.
Innamorati, M., Pompili, M., Serafini, G., Lester, D., Erbuto, D., Amore, M., et al. (2011). Psychometric properties of the Suicidal History Self-rating Screening Scale. Archives of Suicide Research, 15(1), 87–92. doi:10.1080/13811118.2011.540471.
Jobes, D. A. (2006). Managing suicidal risk: A collaborative approach. New York, NY: Guilford Press.
Jobes, D. A., Nelson, K. N., Peterson, E. M., Pentiuc, D., Downing, V., Francini, K., & Kiernan, A. (2004). Describing suicidality: An investigation of qualitative SSF responses. Suicide Life Threat Behav, 34(2), 99–112. doi:10.1521/suli.34.2.99.32788
Johnson, J., Gooding, P., & Tarrier, N. (2008). Suicide risk in schizophrenia: Explanatory models and clinical implications, The Schematic Appraisal Model of Suicide (SAMS). Psychol Psychother Theory Res Pract, 81(1), 55–77. doi:10.1348/147608307X244996.
Kessler, R. C., & Üstün, T. B. (2004). The World Mental Health (WMH) Survey Initiative version of the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI). Int J Methods Psychiatr Res, 13(2), 93–117. doi:10.1002/mpr.168.
Kessler, R. C., Borges, G., & Walters, E. E. (1999). Prevalence of and risk factors for lifetime suicide attempts in the National Comorbidity Survey. Arch Gen Psychiatry, 56(7), 617–626.
King, C. A., Eisenberg, D., Zheng, K., Czyz, E., Kramer, A., Horwitz, A., & Chermack, S. (2015). Online suicide risk screening and intervention with college students: A pilot randomized controlled trial. J Consult Clin Psychol, 83(3), 630–636. doi:10.1037/a0038805.
Klonsky, E. D., & May, A. M. (2014). Differentiating suicide attempters from suicide ideators: A critical frontier for suicidology research. Suicide Life Threat Behav, 44(1), 1–5. doi:10.1111/sltb.12068.
Kovacs, M., & Beck, A. T. (1977). The wish to die and the wish to live in attempted suicides. J Clin Psychol, 33(2), 361–365.
Kovacs, M., Beck, A. T., & Weissman, A. (1976). The communication of suicidal intent: A reexamination. Arch Gen Psychiatry, 33(2), 198–201.
Kral, M. J., & Sakinofsky, I. (1994). Clinical model for suicide risk assessment. Death Studies, 18(4), 311–326.
Kumagai, R. (2012). A new method for estimating differential item functioning (DIF) for multiple groups and polytomous items: Development of index K and the computer program "EasyDIF". Japanese Journal of Psychology, 83(1), 35–43.
Kumar, C. T. S., Mohan, R., Ranjith, G., & Chandrasekaran, R. (2006). Characteristics of high intent suicide attempters admitted to a general hospital. J Affect Disord, 91(1), 77–81. doi:10.1016/j.jad.2005.12.028.
Langhinrichsen-Rohling, J., Snarr, J. D., Slep, A. M. S., Heyman, R. E., & Foran, H. M. (2011). Risk for suicidal ideation in the U.S. Air Force: An ecological perspective. J Consult Clin Psychol, 79(5), 600–612. doi:10.1037/a0024631.
Lecrubier, Y., Sheehan, D. V., Weiller, E., Amorim, P., Bonora, I., Sheehan, K. H., et al. (1997). The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: Reliability and validity according to the CIDI. Eur Psychol, 12(5), 224–231. doi:10.1016/S0924–9338(97)83296–8.
Letizia, C., Kapik, B., & Flanders, W. D. (1996). Suicidal risk during controlled clinical investigations of fluvoxamine. J Clin Psychiatry, 57(9), 415–421.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychol Methods, 7(1), 19–40. doi:10.1037//1082-989X.7.1.19.
Markon, K. E., Chmielewski, M., & Miller, C. J. (2011). The reliability and validity of discrete and continuous measures of psychopathology: A quantitative review. Psychol Bull, 137(5), 856–879. doi:10.1037/a0023678.
Mazerolle, M. J. (2015). AICcmodavg: Model selection and multimodel inference based on (Q)AIC(c) . Retrieved from http://CRAN.R-project.org/package=AICcmodavg.
McClain, C. S., Rosenfeld, B., & Breitbart, W. (2003). Effect of spiritual well-being on end-of-life despair in terminally-ill cancer patients. Lancet, 361(9369), 1603–1607. doi:10.1016/s0140–6736(03)13310–7.
Meijer, R. R., & Egberink, I. J. L. (2012). Investigating invariant item ordering in personality and clinical scales: Some empirical findings and a discussion. Educ Psychol Meas, 72(4), 589–607. doi:10.1177/0013164411429344.
Miller, G. K., Piscopo, K. D., Batts, K., Han, B., Colpe, L., Forman-Hoffman, V. L., et al. (2015). Measurement of suicidal thoughts, behaviors, and related health outcomes in the United States: Comparison of NSDUH estimates with other data sources. CBHSQ Data Review: SAMHSA.
Misson, H., Mathieu, F., Jollant, F., Yon, L., Guillaume, S., Parmentier, C., et al. (2010). Factor analyses of the Suicidal Intent Scale (SIS) and the Risk-Rescue Rating Scale (RRRS): Toward the identification of homogeneous subgroups of suicidal behaviors. J Affect Disord, 121(1–2), 80–87. doi:10.1016/j.jad.2009.05.012.
Modai, I., Ritsner, M., Kurs, R., Mendel, S., & Ponizovsky, A. (2002). Validation of the Computerized Suicide Risk Scale - A backpropagation neural network instrument (CSRS-BP). Eur Psychol, 17(2), 75–81.
Muraki, E. (1993). Information functions of the generalized partial credit model. Appl Psychol Meas, 17(4), 351–363.
Nock, M. K., & Kessler, R. C. (2006). Prevalence of and risk factors for suicide attempts versus suicide gestures: Analysis of the National Comorbidity Survey. J Abnorm Psychol, 115(3), 616–623.
Nock, M. K., Holmberg, E. B., Photos, V. I., & Michel, B. D. (2007). Self-Injurious Thoughts and Behaviors Interview: Development, reliability, and validity in an adolescent sample. Psychol Assess, 19(3), 309–317.
Nock, M. K., Borges, G., Bromet, E. J., Cha, C. B., Kessler, R. C., & Lee, S. (2008). Suicide and suicidal behavior. Epidemiol Rev, 30(1), 133–154. doi:10.1093/epirev/mxn002.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
Olfson, M., Marcus, S. C., & Bridge, J. A. (2014). Focusing suicide prevention on periods of high risk. J Am Med Assoc, 311(11), 1107–1108. doi:10.1001/jama.2014.501.
Osman, A., Bagge, C. L., Gutierrez, P. M., Konick, L. C., Kopper, B. A., & Barrios, F. X. (2001). The Suicidal Behaviors Questionnaire-Revised (SBQ-R): Validation with clinical and nonclinical samples. Assessment, 8(4), 443–454.
Osterlin, S. J., & Everson, H. T. (2009). Differential item functioning (2nd ed.). London: SAGE Publications.
Patterson, W. M., Dohn, H. H., Bird, J., & Patterson, G. A. (1983). Evaluation of suicidal patients: The SAD PERSONS scale. Psychosomatics, 24(4), 343–345 348.
Pfeffer, C. R., Conte, H. R., Plutchik, R., & Jerrett, I. (1979). Suicidal behavior in latency-age children. An empirical study. Journal of the American Academy of Child Psychiatry, 18(4), 679–692.
Plöderl, M., Kralovec, K., Yazdi, K., & Fartacek, R. (2011). A closer look at self-reported suicide attempts: False positives and false negatives. Suicide Life Threat Behav, 41(1), 1–5.
Posner, K., Brown, G. K., Stanley, B., Brent, D. A., Yershova, K. V., Oquendo, M. A., et al. (2011). The Columbia-Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatr, 168(12), 1266–1277. doi:10.1176/appi.ajp.2011.10111704.
Range, L. M. (2005). The family of instruments that assess suicide risk. J Psychopathol Behav Assess, 27(2), 133–140. doi:10.1007/s10862–005–5387-8.
Range, L. M., & Knott, E. C. (1997). Twenty suicide assessment instruments: Evaluation and recommendations. Death Studies, 21(1), 25–58. doi:10.1080/074811897202128.
Revelle, W. (2016). psych: Procedures for personality and psychological research (Version 1.6.3) [R package version 1.4.12]. Evanston, IL: Northwestern University . Retrieved from http://cran.r-project.org/web/packages/psych/.
Rizopoulos, D. (2006). ltm: An R package for latent variable modelling and item response theory analyses. J Stat Softw, 17(5), 1–25.
Rogers, J. R., Lewis, M. M., & Subich, L. M. (2002). Validity of the Suicide Assessment Checklist in an emergency crisis center. J Couns Dev, 80(4), 493–502.
Rudd, M. D. (2006). Fluid vulnerability theory: A cognitive approach to understanding the process of acute and chronic suicide risk. In T. E. Ellis (Ed.), Cognition and suicide: Theory, research, and therapy (pp. 355–368). Washington, DC: American Psychological Association.
Rudd, M. D., Cukrowicz, K. C., & Bryan, C. J. (2008). Core competencies in suicide risk assessment and management: Implications for supervision. Training and Education in Professional Psychology, 2(4), 219–228. doi:10.1037/1931–3918.2.4.219
Sapyta, J., Goldston, D. B., Erkanli, A., Daniel, S. S., Heilbron, N., Mayfield, A., & Lyn Treadway, S. (2012). Evaluating the predictive validity of suicidal intent and medical lethality in youth. J Consult Clin Psychol, 80(2), 222–231. doi:10.1037/a0026870.
Schmitz, W. M., Allen, M. H., Feldman, B. N., Gutin, N. J., Jahn, D. R., Kleespies, P. M., et al. (2012). Preventing suicide through improved training in suicide risk assessment and care: An American Association of Suicidology task force report addressing serious gaps in U.S. mental health training. Suicide Life Threat Behav, 42(3), 292–304. doi:10.1111/j.1943-278X.2012.00090.x.
Shneidman, E. S. (1996). The suicidal mind. New York: Oxford University Press.
Shneidman, E. S. (2005). Anodyne psychotherapy for suicide: a psychological view of suicide. Clinical Neuropsychiatry: Journal of Treatment Evaluation, 2(1), 7–12.
Simon, G. E., Savarino, J., Operskalski, B., & Wang, P. S. (2006). Suicide risk during antidepressent treatment. Am J Psychiatr, 163(1), 41–47.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education.
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychol Bull, 133(5), 859–883. doi:10.1037/0033–2909.133.5.859
Venta, A., & Sharp, C. (2014). Extending the concurrent validity of the Self-Injurious Thoughts and Behaviors Interview to inpatient adolescents. J Psychopathol Behav Assess, 36(4), 675–682. doi:10.1007/s10862–013–9402-1.
Viglione, D., Giromini, L., Gustafson, M. L., & Meyer, G. J. (2014). Developing continuous variable composites for Rorschach measures of thought problems, vigilance, and suicide risk. Assessment, 21(1), 42–49. doi:10.1177/1073191112446963.
Wagenmakers, E. J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychon Bull Rev, 11(1), 192–196.
World Health Organization (2014). Preventing suicide: A global imperative. Geneva, Switzerland: WHO Press.
Acknowledgments
No funding was received for this project.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
Keith M. Harris, Owen D. Lello and Christopher H. Willcox report no conflict of interest related to this study.
Experiment Participants
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all participants for the voluntary and anonymous surveys of the studies. All studies were approved by the appropriate institutional review board of the host institution.
Ethical Approval
This article does not contain any studies with animals performed by any of the authors.
Electronic supplementary material
ESM 1
(DOCX 12 kb)
Rights and permissions
About this article
Cite this article
Harris, K.M., Lello, O.D. & Willcox, C.H. Reevaluating Suicidal Behaviors: Comparing Assessment Methods to Improve Risk Evaluations. J Psychopathol Behav Assess 39, 128–139 (2017). https://doi.org/10.1007/s10862-016-9566-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-016-9566-6