European Child & Adolescent Psychiatry

, Volume 27, Issue 2, pp 159–169 | Cite as

Assessing for suicidal behavior in youth using the Achenbach System of Empirically Based Assessment

  • Anna R. Van MeterEmail author
  • Guillermo Perez Algorta
  • Eric A. Youngstrom
  • Yana Lechtman
  • Jen K. Youngstrom
  • Norah C. Feeny
  • Robert L. Findling
Original Contribution


This study investigated the clinical utility of the Achenbach System of Empirically Based Assessment (ASEBA) for identifying youth at risk for suicide. Specifically, we investigated how well the Total Problems scores and the sum of two suicide-related items (#18 “Deliberately harms self or attempts suicide” and #91 “Talks about killing self”) were able to distinguish youth with a history of suicidal behavior. Youth (N = 1117) aged 5–18 were recruited for two studies of mental illness. History of suicidal behavior was assessed by semi-structured interviews (K-SADS) with youth and caregivers. Youth, caregivers, and a primary teacher each completed the appropriate form (YSR, CBCL, and TRF, respectively) of the ASEBA. Areas under the curve (AUCs) from ROC analyses and diagnostic likelihood ratios (DLRs) were used to measure the ability of both Total Problems T scores, as well as the summed score of two suicide-related items, to identify youth with a history of suicidal behavior. The Suicide Items from the CBCL and YSR performed well (AUCs = 0.85 and 0.70, respectively). The TRF Suicide Items did not perform better than chance, AUC = 0.45. The AUCs for the Total Problems scores were poor-to-fair (0.33–0.65). The CBCL Suicide Items outperformed all other scores (ps = 0.04 to <0.0005). Combining the CBCL and YSR items did not lead to incremental improvement in prediction over the CBCL alone. The sum of two questions from a commonly used assessment tool can offer important information about a youth’s risk for suicidal behavior. The low burden of this approach could facilitate wide-spread screening for suicide in an increasingly at-risk population.


Suicide Diagnostic accuracy ROC Evidence-based assessment 


Compliance with ethical standards

Conflict of interest

Dr. Feeny has no conflict of interest to report. Dr. Findling receives or has received research support, acted as a consultant and/or served on a speaker’s bureau for Actavis, Akili, Alcobra, American Academy of Child & Adolescent Psychiatry, American Psychiatric Press, Bracket, CogCubed, Cognition Group, Coronado Biosciences, Elsevier, Epharma Solutions, Forest, Genentech, GlaxoSmithKline, Guilford Press, Ironshore, Johns Hopkins University Press, KemPharm, Lundbeck, Medgenics, Merck, NIH, Neurim, Novartis, Otsuka, PCORI, Pfizer, Physicians Postgraduate Press, Purdue, Rhodes Pharmaceuticals, Roche, Sage, Shire, Sunovion, Supernus Pharmaceuticals, Syneurx, Takeda, Teva, TouchPoint, Tris, Validus, and WebMD. Ms. Lechtman has no conflict of interest to report. Dr. Perez Algorta has no conflict of interest to report. Dr. Van Meter has no conflict of interest to report. Dr. J. Youngstrom has no conflict of interest to report. Dr. E. Youngstrom has consulted with Pearson, Janssen, Otsuka, Lundbeck, and Western Psychological Services about psychological assessment.

Supplementary material

787_2017_1030_MOESM1_ESM.docx (79 kb)
Supplementary material 1 (DOCX 79 kb)
787_2017_1030_MOESM2_ESM.docx (64 kb)
Supplementary material 2 (DOCX 64 kb)


  1. 1.
    Centers for Disease Control (2015) Youth suicide. Accessed Jan 8, 2016
  2. 2.
    Curtin S, Warner M, Hedegaard H (2016) Increase in suicide in the United States, 1999–2014. National Center for Health Statistics. Accessed May 11, 2016
  3. 3.
    Nock M, Green J, Hwang I, McLaughlin I, Sampson N, Zaslavsky A, Kessler R (2013) Prevalence, correlates, and treatment of lifetime suicidal behavior among adolescents: results from the national comorbidity survey replication adolescent supplement. JAMA Psychiatry 70(3):300–310. doi: 10.1001/2013.jamapsychiatry.55 CrossRefPubMedGoogle Scholar
  4. 4.
    Nock M, Borges G, Bromet E, Cha C, Kessler R, Lee S (2008) Suicide and suicidal behavior. Epidemiol Rev 30:133–154. doi: 10.1093/epirev/mxn002 CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Bridge JA, Goldstein TR, Brent DA (2006) Adolescent suicide and suicidal behavior. J Child Psychol Psychiatry 47(3–4):372–394. doi: 10.1111/j.1469-7610.2006.01615.x CrossRefPubMedGoogle Scholar
  6. 6.
    Cash SJ, Bridge JA (2009) Epidemiology of youth suicide and suicidal behavior. Curr Opin Pediatr 21(5):613CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    World Health Organization (2009) Global health risks: mortality and burden of disease attributable to selected major risks. WHO Press, GenevaGoogle Scholar
  8. 8.
    Joiner TE (1999) The clustering and contagion of suicide. Curr Dir Psychol Sci 8(3):89–92. doi: 10.1111/1467-8721.00021 CrossRefGoogle Scholar
  9. 9.
    Randall JR, Nickel NC, Colman I (2015) Contagion from peer suicidal behavior in a representative sample of American adolescents. J Affect Disord 186:219–225. doi: 10.1016/j.jad.2015.07.001 CrossRefPubMedGoogle Scholar
  10. 10.
    Dilillo D, Mauri S, Mantegazza C, Fabiano V, Mameli C, Zuccotti GV (2015) Suicide in pediatrics: epidemiology, risk factors, warning signs and the role of the pediatrician in detecting them. Ital J Pediatr 41:49. doi: 10.1186/s13052-015-0153-3 CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Ribeiro JD, Franklin JC, Fox KR, Bentley KH, Kleiman EM, Chang BP, Nock MK (2016) Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: a meta-analysis of longitudinal studies. Psychol Med 46(02):225–236. doi: 10.1017/S0033291715001804 CrossRefPubMedGoogle Scholar
  12. 12.
    Helms SW, Prinstein MJ (2014) Risk assessment and decision making regarding imminent suicidality in pediatric settings. Clin Pract Pediatr Psychol 2(2):176CrossRefGoogle Scholar
  13. 13.
    Youngstrom EA (2015) Using standardized methods to assess suicidal behavior: the need is even greater than it looks (Commentary). J Clin Psychiatry 76(10):e1331–e1332. doi: 10.4088/JCP.14com09573 CrossRefPubMedGoogle Scholar
  14. 14.
    Qin P, Nordentoft M (2005) Suicide risk in relation to psychiatric hospitalization: evidence based on longitudinal registers. Arch Gen Psychiatry 62(4):427–432. doi: 10.1001/archpsyc.62.4.427 CrossRefPubMedGoogle Scholar
  15. 15.
    Dawes R, Faust D, Meehl P (1989) Clinical versus actuarial judgment. Science 243(4899):1668–1674. doi: 10.1126/science.2648573 CrossRefPubMedGoogle Scholar
  16. 16.
    Grove WM, Zald DH, Lebow BS, Snitz BE, Nelson C (2000) Clinical versus mechanical prediction: a meta-analysis. Psychol Assess 12(1):19–30. doi: 10.1037/1040-3590.12.1.19 CrossRefPubMedGoogle Scholar
  17. 17.
    Aegisdottir S, White MJ, Spengler PM, Maugherman AS, Anderson LA, Cook RS, Nichols CN, Lampropoulos GK, Walker BS, Cohen G, Rush JD (2006) The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction. Couns Psychol 34(3):341–382. doi: 10.1177/0011000005285875 CrossRefGoogle Scholar
  18. 18.
    Croskerry P (2009) Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv Health Sci Educ 14:27–35. doi: 10.1007/s10459-009-9182-2 CrossRefGoogle Scholar
  19. 19.
    Lilienfeld SO, Lynn SJ (2014) Errors/biases in clinical decision making. In: the encyclopedia of clinical psychology. Wiley. doi:  10.1002/9781118625392.wbecp567
  20. 20.
    Jenkins MM, Youngstrom EA (2016) A randomized controlled trial of cognitive debiasing improves assessment and treatment selection for pediatric bipolar disorder. J Consult Clin Psychol. doi: 10.1037/ccp0000070 PubMedPubMedCentralGoogle Scholar
  21. 21.
    Van Meter A, Youngstrom E, Youngstrom J, Feeny N, Findling R (2011) Examining the validity of cyclothymic disorder in a youth sample. J Affect Disord 132(1–2):55–63. doi: 10.1016/j.jad.2011.02.004 CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Youngstrom EA (2012) Future directions in psychological assessment: combining evidence-based medicine innovations with psychology’s historical strengths to enhance utility. J Clin Child Adolesc Psychol 42(1):139–159. doi: 10.1080/15374416.2012.736358 CrossRefPubMedGoogle Scholar
  23. 23.
    Youngstrom EA, Choukas-Bradley S, Calhoun CD, Jensen-Doss A (2014) Clinical guide to the evidence-based assessment approach to diagnosis and treatment. Cognit Behav Pract. doi: 10.1016/j.cbpra.2013.12.005 Google Scholar
  24. 24.
    Lambert MJ, Hansen NB, Finch AE (2001) Patient-focused research: using patient outcome data to enhance treatment effects. J Consult Clin Psychol 69(2):159–172. doi: 10.1037/0022-006x.69.2.159 CrossRefPubMedGoogle Scholar
  25. 25.
    Christon LM, McLeod BD, Jensen-Doss A (2014) Evidence-Based Assessment Meets Evidence-Based Treatment: An Approach to Science-Informed Case Conceptualization. Cognit Behav Pract. doi: 10.1016/j.cbpra.2013.12.004 Google Scholar
  26. 26.
    Straus SE, Glasziou P, Richardson WS, Haynes RB (2011) Evidence-based medicine: How to practice and teach EBM, 4th edn. Churchill Livingstone, New YorkGoogle Scholar
  27. 27.
    Hamilton JD (2001) Do we under utilise actuarial judgement and decision analysis? Evid Based Ment Health 4(4):102–103. doi: 10.1136/ebmh.4.4.102 CrossRefPubMedGoogle Scholar
  28. 28.
    Jenkins M, Youngstrom E, Washburn J, Youngstrom J (2011) Evidence-based strategies improve assessment of pediatric bipolar disorder by community practitioners. Prof Psychol Res Pract 42(2):121CrossRefGoogle Scholar
  29. 29.
    Achenbach T, Rescorla L (2001) Manual for the ASEBA school-age forms and profiles. Research Center for Children, Youth, and Families, BurlingtonGoogle Scholar
  30. 30.
    Pauschardt J, Remschmidt H, Mattejat F (2010) Assessing child and adolescent anxiety in psychiatric samples with the Child Behavior Checklist. J Anxiety Disord 24(5):461–467. doi: 10.1016/j.janxdis.2010.03.002 CrossRefPubMedGoogle Scholar
  31. 31.
    Achenbach TM (2005) Advancing assessment of children and adolescents: commentary on evidence-based assessment of child and adolescent disorders. J Clin Child Adolesc Psychol 34(3):541–547. doi: 10.1207/s15374424jccp3403_9 CrossRefPubMedGoogle Scholar
  32. 32.
    Jones JE, Siddarth P, Gurbani S, Shields WD, Caplan R (2013) Screening for suicidal ideation in children with epilepsy. Epilepsy Behav 29(3):521–526. doi: 10.1016/j.yebeh.2013.09.020 CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    Ferdinand RF, van der Ende J, Verhulst FC (2004) Parent-adolescent disagreement regarding psychopathology in adolescents from the general population as a risk factor for adverse outcome. J Abnorm Psychol 113(2):198–206. doi: 10.1037/0021-843X.113.2.198 CrossRefPubMedGoogle Scholar
  34. 34.
    Youngstrom E, Meyers O, Demeter C, Youngstrom J, Morello L, Piiparinen R, Feeny N, Calabrese J, Findling R (2005) Comparing diagnostic checklists for pediatric bipolar disorder in academic and community mental health settings. Bipolar Disord 7:507–517. doi: 10.1111/j.1399-5618.2005.00269.x CrossRefPubMedGoogle Scholar
  35. 35.
    GrØholt B, Ekeberg Ø, WichstrØm L, Haldorsen T (1998) Suicide among children and younger and older adolescents in Norway: a comparative study. J Am Acad Child Adolesc Psychiatry 37(5):473–481. doi: 10.1097/00004583-199805000-00008 CrossRefPubMedGoogle Scholar
  36. 36.
    Brent DA, Baugher M, Bridge J, Chen T, Chiappetta L (1999) Age- and sex-related risk factors for adolescent suicide. J Am Acad Child Adolesc Psychiatry 38(12):1497–1505. doi: 10.1097/00004583-199912000-00010 CrossRefPubMedGoogle Scholar
  37. 37.
    Youngstrom E, Findling R, Calabrese J, Gracious B, Demeter C, DelPorto Bedoya D, Price M (2004) Comparing the diagnostic accuracy of six potential screening instruments for bipolar disorder in youths aged 5 to 17 years. J Am Acad Child Adolesc Psychiatry 43(7):847–858. doi: 10.1097/01.chi.0000125091.35109.1e CrossRefPubMedGoogle Scholar
  38. 38.
    Findling R, Youngstrom E, McNamara N, Stansbrey R, Demeter C, Bedoya D, Kahana S, Calabrese J (2005) Early symptoms of mania and the role of parental risk. Bipolar Disord 7(6):623–634. doi: 10.1111/j.1399-5618.2005.00260.x CrossRefPubMedGoogle Scholar
  39. 39.
    Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, Williamson D, Ryan N (1997) Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry 36(7):980–988. doi: 10.1097/00004583-199707000-00021 CrossRefPubMedGoogle Scholar
  40. 40.
    Geller B, Zimerman B, Williams M, Bolhofner K, Craney JL, DelBello MP, Soutullo C (2001) Reliability of the Washington University in St. Louis Kiddie schedule for affective disorders and schizophrenia (WASH-U-KSADS) mania and rapid cycling sections. J Am Acad Child Adolesc Psychiatry 40(4):450–455CrossRefPubMedGoogle Scholar
  41. 41.
    Achenbach T (1991) Manual for the child behavior checklist. University of Vermont, Department of Psychiatry, BurlingtonGoogle Scholar
  42. 42.
    Achenbach T (1991) Manual for the youth self-report and 1991 profile. Department of Psychiatry, University of Vermont, VermontGoogle Scholar
  43. 43.
    Achenbach T (1991) Teacher report form. Department of Psychiatry, University of Vermont, BurlingtonGoogle Scholar
  44. 44.
    Spitzer R (1983) Psychiatric diagnosis: are clinicians still necessary? Compr Psychiatry 24(5):399–411. doi: 10.1016/0010-440X(83)90032-9 CrossRefPubMedGoogle Scholar
  45. 45.
    Venkatraman ES (2000) A permutation test to compare receiver operating characteristic curves. Biometrics 56(4):1134–1138. doi: 10.2307/2677047 CrossRefPubMedGoogle Scholar
  46. 46.
    Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, Muller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform 12:77. doi: 10.1186/1471-2105-12-77 CrossRefGoogle Scholar
  47. 47.
    Horowitz LM, Bridge JA, Teach SJ et al (2012) Ask suicide-screening questions (asq): a brief instrument for the pediatric emergency department. Arch Pediatr Adolesc Med 166(12):1170–1176. doi: 10.1001/archpediatrics.2012.1276 CrossRefPubMedGoogle Scholar
  48. 48.
    Joiner TE, Rudd MD, Rajab MH (1999) Agreement between self- and clinician-rated suicidal symptoms in a clinical sample of young adults: explaining discrepancies. J Consult Clin Psychol 67(2):171–176. doi: 10.1037/0022-006X.67.2.171 CrossRefPubMedGoogle Scholar
  49. 49.
    Kaplan ML, Asnis GM, Sanderson WC, Keswani L, de Lecuona JM, Joseph S (1994) Suicide assessment: clinical interview vs. self-report. J Clin Psychol 50(2):294–298. doi: 10.1002/1097-4679(199403)50:2<294:AID-JCLP2270500224>3.0.CO;2-R CrossRefPubMedGoogle Scholar
  50. 50.
    Bridge JA, Barbe RP, Birmaher B, Kolko DJ, Brent DA (2005) Emergent suicidality in a clinical psychotherapy trial for adolescent depression. Am J Psychiatry 162(11):2173–2175. doi: 10.1176/appi.ajp.162.11.2173 CrossRefPubMedGoogle Scholar
  51. 51.
    Shaffer D, Scott M, Wilcox H, Maslow C, Hicks R, Lucas CP, Garfinkel R, Greenwald S (2004) The Columbia SuicideScreen: validity and reliability of a screen for youth suicide and depression. J Am Acad Child Adolesc Psychiatry 43(1):71–79. doi: 10.1097/00004583-200401000-00016 CrossRefPubMedGoogle Scholar
  52. 52.
    Youngstrom EA, Van Meter A (2016) Empirically supported assessment of children and adolescents. Clin Psychol Sci Pract 23(4):327–347. doi: 10.1111/cpsp.12172 CrossRefGoogle Scholar
  53. 53.
    Van Meter A, Youngstrom E, Youngstrom JK, Ollendick T, Demeter C, Findling RL (2014) Clinical decision making about child and adolescent anxiety disorders using the achenbach system of empirically based assessment. J Clin Child Adolesc Psychol. doi: 10.1080/15374416.2014.883930 PubMedPubMedCentralGoogle Scholar
  54. 54.
    Straus SE, Glasziou P, Richardson WS, Haynes RB (2011) Evidence-based medicine: how to practice and teach it, 4th edn. Elsevier, LondonGoogle Scholar
  55. 55.
    Meyer RE, Salzman C, Youngstrom EA, Clayton PJ, Goodwin FK, Mann JJ, Alphs LD, Broich K, Goodman WK, Greden JF, Meltzer HY, Normand S-LT, Posner K, Shaffer D, Oquendo MA, Stanley B, Trivedi MH, Turecki G, Beasley CM, Beautrais AL, Bridge JA, Brown GK, Revicki DA, Ryan ND, Sheehan DV (2010) Suicidality and risk of suicide—definition, drug safety concerns, and a necessary target for drug development. J Clin Psychiatry 71(08):e1–e21. doi: 10.4088/JCP.10cs06070blu CrossRefPubMedGoogle Scholar
  56. 56.
    Horowitz LM, Bridge JA, Pao M, Boudreaux ED (2014) Screening youth for suicide risk in medical settings: time to ask questions. American Journal of Preventive Medicine 47(3, Supplement 2):S170–S175. doi: 10.1016/j.amepre.2014.06.002 CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Anna R. Van Meter
    • 1
    Email author
  • Guillermo Perez Algorta
    • 2
  • Eric A. Youngstrom
    • 3
  • Yana Lechtman
    • 1
  • Jen K. Youngstrom
    • 3
  • Norah C. Feeny
    • 4
  • Robert L. Findling
    • 5
  1. 1.Ferkauf Graduate SchoolYeshiva UniversityBronxUSA
  2. 2.Lancaster UniversityLancasterUK
  3. 3.University of North CarolinaChapel HillUSA
  4. 4.Case Western Reserve UniversityClevelandUSA
  5. 5.Johns Hopkins University/Kennedy Krieger InstituteBaltimoreUSA

Personalised recommendations