Skip to main content
SpringerLink
Log in

Automated Risk Assessment for School Violence: a Pilot Study

Psychiatric Quarterly volume 89, pages 817–828 (2018)Cite this article

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

School violence has increased over the past ten years. This study evaluated students using a more standard and sensitive method to help identify students who are at high risk for school violence. 103 participants were recruited through Cincinnati Children’s Hospital Medical Center (CCHMC) from psychiatry outpatient clinics, the inpatient units, and the emergency department. Participants (ages 12–18) were active students in 74 traditional schools (i.e. non-online education). Collateral information was gathered from guardians before participants were evaluated. School risk evaluations were performed with each participant, and audio recordings from the evaluations were later transcribed and manually annotated. The BRACHA (School Version) and the School Safety Scale (SSS), both 14-item scales, were used. A template of open-ended questions was also used. This analysis included 103 participants who were recruited from 74 different schools. Of the 103 students evaluated, 55 were found to be moderate to high risk and 48 were found to be low risk based on the paper risk assessments including the BRACHA and SSS. Both the BRACHA and the SSS were highly correlated with risk of violence to others (Pearson correlations>0.82). There were significant differences in BRACHA and SSS total scores between low risk and high risk to others groups (p-values <0.001 under unpaired t-test). In particular, there were significant differences in individual SSS items between the two groups (p-value <0.001). Of these items, Previous Violent Behavior (Pearson Correlation = 0.80), Impulsivity (0.69), School Problems (0.64), and Negative Attitudes (0.61) were positively correlated with risk to others. The novel machine learning algorithm achieved an AUC of 91.02% when using the interview content to predict risk of school violence, and the AUC increased to 91.45% when demographic and socioeconomic data were added. Our study indicates that the BRACHA and SSS are clinically useful for assessing risk for school violence. The machine learning algorithm was highly accurate in assessing school violence risk.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

References

  1. Musu-Gillette, L., Zhang, A., Wang, K., Zhang, J., and Oudekerk, B.A. (2017). Indicators of school crime and safety: 2016. National Center for Education Statistics, U.S. Department of Education, and Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice. Washington, DC.

  2. Centers for Disease Control and Prevention (CDC), 1992–2014 School-Associated Violent Death Surveillance System (SAVD-SS), retrieved July 2016 from http://www.cdc.gov/injury/wisqars/index.html; and Federal Bureau of Investigation and Bureau of Justice Statistics, Supplementary Homicide Reports (SHR), preliminary data (August 2016).

  3. National Association of School Psychologists. (2010). Crisis and safety resources. Retrieved April 3, 2014 from http://www.nasponline.org/educators/index.aspx#crisis.

  4. McCoy DC, Roy AL, Sirkman GM. Neighborhood crime and school climate as predictors of elementary school academic quality: a cross-lagged panel analysis. Am J Community Psychol. 2013 Sep;52(1–2):128–40. https://doi.org/10.1007/s10464-013-9583-5.

    Article  PubMed  Google Scholar 

  5. Burdick-Will J. School violent crime and academic achievement in Chicago. Sociol Educ. 2013 Oct;86(4):343–61. https://doi.org/10.1177/0038040713494225.

    Article  Google Scholar 

  6. Strøm IF, Thoresen S, Wentzel-Larsen T, Dyb G. Violence, bullying and academic achievement: a study of 15-year-old adolescents and their school environment. Child Abuse Negl 2013 Apr;37(4):243–251. doi: https://doi.org/10.1016/j.chiabu.2012.10.010. Epub 2013 Jan 6.

    Article  Google Scholar 

  7. Gottfredson GD, Cook PJ, NA C. Schools and prevention. In: Welsh BC, Farrington DP, editors. Crime and prevention. Oxford, United Kingdom: Oxford University Press; 2000. p. 269–87.

    Google Scholar 

  8. Tanner-Smith EE, Wilson SJ, Lipsey MW. Risk factors and crime. In: Maguire M, Morgan R, Reiner R, editors. The Oxford handbook of criminology. 5th edn. Oxford: Oxford University Press; 2012. p. 89–111.

    Google Scholar 

  9. Mytton J, DiGuiseppi C, Gough D, Taylor R, Logan S. School-based secondary prevention programmes for preventing violence. Cochrane Database Syst Rev. 2006 Jul 19;3 CD004606

  10. Park-Higgerson HK, Perumean-Chaney SE, Bartolucci AA, Grimley DM, Singh KP. The evaluation of school-based violence prevention programs: a meta-analysis. J Sch Health 2008 Sep;78(9):465–479; quiz 518-20. doi: https://doi.org/10.1111/j.1746-1561.2008.00332.x.

    Article  PubMed  Google Scholar 

  11. Borum R, Cornell DG, Modzeleski W, Jimerson SR. What can be done about school shootings? A review of the evidence. Educ Res. 2010;39(1):27–37.

    Article  Google Scholar 

  12. Nekvasil EK, Cornell DG. Student reports of peer threats of violence: prevalence and outcomes. J Sch Violence. 2012;11(4):357–75.

    Article  Google Scholar 

  13. Bernes KB, Bardick AD. Conducting adolescent violence risk assessments: a framework for school counselors. Prof Sch Couns. 2007;10(4):419–27.

    Article  Google Scholar 

  14. McGowan MR, Horn RA, Mellott RN. The predictive validity of the structured assessment of violence risk in youth in secondary educational settings. Psychol Assess. 2011;23(2):478–86.

    Article  PubMed  Google Scholar 

  15. Monahan J, Steadman H. Violence risk assessment: a quarter century of research. In: Frost L, Bonnie R, editors. The evolution of mental health law. Washington: American Psychological Association; 2001. p. 195–211. https://doi.org/10.1037/10414-010.

    Chapter  Google Scholar 

  16. Barzman D, Brackenbury L, Sonnier L, Schnell B, Cassedy A, Salisbury S, et al. Brief rating of aggression by children and adolescents (BRACHA): development of a tool to assess risk of inpatients’ aggressive behavior. J Am Acad Psychiatry Law. 2011;39(2):170–9.

    PubMed  Google Scholar 

  17. Xia F, Yetisgen-Yildiz: Clinical corpus annotation: challenges and strategies. Proc. Of Third Workshop on Building and Evaluating Resources for Biomedical Text Mining of the International Conference on Language Resources and Evaluation, 2012.

  18. Kors JA, Clematide S, Akhondi SA, van Mulligen EM, Rebholz-Schuhmann D. A multilingual gold-standard corpus for biomedical concept recognition: the mantra GSC. J Am Med Inform Assoc. 2015 Sep;22(5):948–56. https://doi.org/10.1093/jamia/ocv037.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Wilbur WJ, Rzhetsky A, Shatkay H. New directions in biomedical text annotation: definitions, guidelines and corpus construction. BMC Bioinformatics. 2006 Jul 25;7:356.

    Article  PubMed Central  PubMed  Google Scholar 

  20. Barzman D, Mossman D, Sonnier L, Sorter M. Brief rating of aggression by children and adolescents (BRACHA): a reliability study. J Am Acad Psychiatry Law. 2012;40:374–82.

    PubMed  Google Scholar 

  21. Barzman DH, Ni Y, Griffey M, Patel B, Warren A, Latessa E, et al. A pilot study on developing a standardized and sensitive school violence risk assessment with manual annotation. Psychiatry Q. 2017;88(3):447–57.

    Article  Google Scholar 

  22. Douglas KS, Blanchard AJE, Guy LS, Reeves KA, Weir J (2010). HCR-20 Violence Risk Assessment Scheme: Overview and Annotated Bibliography. Retrieved from http://kdouglas.files.wordpress.com/2007/10/hcr-20-annotated-biblio-sept-2010.pdf.

  23. Delgado SV, Barzman D, Gehle M, Caring M, Sorter MD, Kowatch R, et al. Characteristics of discharges against medical advice from acute inpatient psychiatric units for children and adolescents. Boston: Poster presented at the annual meeting of the American Academy of Child and Adolescent Psychiatry; 2007.

    Google Scholar 

  24. Hilterman EL, Nicholls TL, van Nieuwenhuizen C: Predictive performance of risk assessments in juvenile offenders: comparing the SAVRY, PCL:YV, and YLS/CMI with unstructured clinical assessments. Assessment, 2014, 21, 324, 339.

  25. Federal Bureau of Investigation. (1999). The school shooter: a threat assessment perspective. (Federal Bureau of Investigation, ED446352). Quantico VA. Retrieved from http://www.fbi.gov/library/school/school2.pdf.

  26. Lingren T, Deleger L, Molnar K, Zhai H, Meinzen-Derr J, Kaiser M, et al. Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. J Am Med Inform Assoc. 2014;2013 https://doi.org/10.1136/amiajnl-2013-001837.

    Article  PubMed  Google Scholar 

  27. Deleger L, Molnar K, Savova G, Xia F, Lingren T, Li Q, et al. Large-scale evaluation of automated clinical note de-identification and its impact on information extraction. J Am Med Inform Assoc. 2012;20(1):84–94.

    Article  PubMed  Google Scholar 

  28. Ganzert S, Guttmann J, Kersting K, Kuhlen R, Putensen C, Sydow M, et al. Analysis of respiratory pressure-volume curves in intensive care medicine using inductive machine learning. Artif Intell Med. 2002;26(1):69–86.

    Article  PubMed  Google Scholar 

  29. Zacharaki EI, Wang S, Chawla S, Soo Yoo D, Wolf R, Melhem ER, et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn Reson Med. 2009;62(6):1609–18.

    Article  PubMed Central  PubMed  Google Scholar 

  30. Zrimec, T., & Kononenko, I. (2004). Feasibility analysis of machine learning in medical diagnosis from aura images. In Proc. Int. Conf. KIRLIONICS-98 (Abstracts) (pp. 10–11).

  31. Sara, NB, Halland R, Igel C, Alstrup S. High-school dropout prediction using machine learning: a Danish large-scale study European symposium on artificial neural networks, computational intelligence and machine learning. Bruges (Belgium), 22–24 April 2015.

  32. Welsh JL, Schmidt F, McKinnon L, Chattha HK, Meyers JR A comparative study of adolescent risk assessment instruments: predictive and incremental validity assessment. 2008 Mar;15(1):104–15.

  33. Molnar BE, Cerda M, Roberts AL, Buka SL. Effects of neighborhood resources on aggressive and delinquent behaviors among urban youths. Am J Public Health. 2008;98:1086–93. https://doi.org/10.2105/AJPH.2006.098913.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Reed MO, Jakubovski E, Johnson JA, Bloch MH. Predictor of long-term school-based behavioral outcomes in the multimodal treatment study of children with attention-deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol. 27(4):296–309.

    Article  PubMed Central  PubMed  Google Scholar 

  35. Singh JP, Grann M, Fazel S. A comparative study of violence risk assessment tools: a systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clin Psychol Rev. 2011;31:499–513.

    Article  PubMed  Google Scholar 

  36. Mossman D. Assessing prediction of violence: being accurate about accuracy. J Consult Clin Psychol. 1994;62(4):783–92.

    Article  CAS  PubMed  Google Scholar 

  37. Janofsky JS, Spears S, Neubauer DN. Psychiatrists' accuracy in predicting violent behavior on an inpatient unit. Hosp Community Psychiatry. 1988;39:1090–4.

    CAS  PubMed  Google Scholar 

  38. Neuman Y, Assaf D, Cohen Y, Knoll J. Profiling school shooters: automatic text-based analysis. Front Psych. 2015;6:1–5.

    Google Scholar 

  39. Shultz JM, Cohen AM, Muschert GW, Flores de Apodaca R. Fatal school shootings and the epidemiological context of firearm mortality in the United States. Disaster Health. 2013 Apr-Dec;1(2):84–101.

    Article  PubMed Central  PubMed  Google Scholar 

  40. Flannery DJ, Modzeleski W, Kretschmar JM. Violence and school shootings. Curr Psychiatry Rep. 2013 Jan;15(1):331.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The Park Foundation, CCTST, and Cincinnati Children’s Hospital Medical Center.

Author information

Authors and Affiliations

  1. Cincinnati Children’s Hospital Medical Center (CCHMC), Cincinnati, USA

    Drew Barzman, Yizhao Ni, Marcus Griffey, Alycia Bachtel, Kenneth Lin & Michael Sorter

  2. University of Dayton, Dayton, USA

    Hannah Jackson

  3. University of Cincinnati, Cincinnati, USA

    Melissa DelBello

Authors
  1. Drew Barzman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yizhao Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcus Griffey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alycia Bachtel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kenneth Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hannah Jackson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michael Sorter
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Melissa DelBello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Drew Barzman.

Ethics declarations

Conflict of Interest

All authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barzman, D., Ni, Y., Griffey, M. et al. Automated Risk Assessment for School Violence: a Pilot Study. Psychiatr Q 89, 817–828 (2018). https://doi.org/10.1007/s11126-018-9581-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11126-018-9581-8

Keywords

search

Navigation