Automated Risk Assessment for School Violence: a Pilot Study
School violence has increased over the past ten years. This study evaluated students using a more standard and sensitive method to help identify students who are at high risk for school violence. 103 participants were recruited through Cincinnati Children’s Hospital Medical Center (CCHMC) from psychiatry outpatient clinics, the inpatient units, and the emergency department. Participants (ages 12–18) were active students in 74 traditional schools (i.e. non-online education). Collateral information was gathered from guardians before participants were evaluated. School risk evaluations were performed with each participant, and audio recordings from the evaluations were later transcribed and manually annotated. The BRACHA (School Version) and the School Safety Scale (SSS), both 14-item scales, were used. A template of open-ended questions was also used. This analysis included 103 participants who were recruited from 74 different schools. Of the 103 students evaluated, 55 were found to be moderate to high risk and 48 were found to be low risk based on the paper risk assessments including the BRACHA and SSS. Both the BRACHA and the SSS were highly correlated with risk of violence to others (Pearson correlations>0.82). There were significant differences in BRACHA and SSS total scores between low risk and high risk to others groups (p-values <0.001 under unpaired t-test). In particular, there were significant differences in individual SSS items between the two groups (p-value <0.001). Of these items, Previous Violent Behavior (Pearson Correlation = 0.80), Impulsivity (0.69), School Problems (0.64), and Negative Attitudes (0.61) were positively correlated with risk to others. The novel machine learning algorithm achieved an AUC of 91.02% when using the interview content to predict risk of school violence, and the AUC increased to 91.45% when demographic and socioeconomic data were added. Our study indicates that the BRACHA and SSS are clinically useful for assessing risk for school violence. The machine learning algorithm was highly accurate in assessing school violence risk.
KeywordsSchool violence School violence risk assessment Natural language processing Machine learning School safety School shootings
The Park Foundation, CCTST, and Cincinnati Children’s Hospital Medical Center.
Compliance with Ethical Standards
Conflict of Interest
All authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- 1.Musu-Gillette, L., Zhang, A., Wang, K., Zhang, J., and Oudekerk, B.A. (2017). Indicators of school crime and safety: 2016. National Center for Education Statistics, U.S. Department of Education, and Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice. Washington, DC.Google Scholar
- 2.Centers for Disease Control and Prevention (CDC), 1992–2014 School-Associated Violent Death Surveillance System (SAVD-SS), retrieved July 2016 from http://www.cdc.gov/injury/wisqars/index.html; and Federal Bureau of Investigation and Bureau of Justice Statistics, Supplementary Homicide Reports (SHR), preliminary data (August 2016).
- 3.National Association of School Psychologists. (2010). Crisis and safety resources. Retrieved April 3, 2014 from http://www.nasponline.org/educators/index.aspx#crisis.
- 7.Gottfredson GD, Cook PJ, NA C. Schools and prevention. In: Welsh BC, Farrington DP, editors. Crime and prevention. Oxford, United Kingdom: Oxford University Press; 2000. p. 269–87.Google Scholar
- 8.Tanner-Smith EE, Wilson SJ, Lipsey MW. Risk factors and crime. In: Maguire M, Morgan R, Reiner R, editors. The Oxford handbook of criminology. 5th edn. Oxford: Oxford University Press; 2012. p. 89–111.Google Scholar
- 9.Mytton J, DiGuiseppi C, Gough D, Taylor R, Logan S. School-based secondary prevention programmes for preventing violence. Cochrane Database Syst Rev. 2006 Jul 19;3 CD004606
- 17.Xia F, Yetisgen-Yildiz: Clinical corpus annotation: challenges and strategies. Proc. Of Third Workshop on Building and Evaluating Resources for Biomedical Text Mining of the International Conference on Language Resources and Evaluation, 2012.Google Scholar
- 22.Douglas KS, Blanchard AJE, Guy LS, Reeves KA, Weir J (2010). HCR-20 Violence Risk Assessment Scheme: Overview and Annotated Bibliography. Retrieved from http://kdouglas.files.wordpress.com/2007/10/hcr-20-annotated-biblio-sept-2010.pdf.
- 23.Delgado SV, Barzman D, Gehle M, Caring M, Sorter MD, Kowatch R, et al. Characteristics of discharges against medical advice from acute inpatient psychiatric units for children and adolescents. Boston: Poster presented at the annual meeting of the American Academy of Child and Adolescent Psychiatry; 2007.Google Scholar
- 24.Hilterman EL, Nicholls TL, van Nieuwenhuizen C: Predictive performance of risk assessments in juvenile offenders: comparing the SAVRY, PCL:YV, and YLS/CMI with unstructured clinical assessments. Assessment, 2014, 21, 324, 339.Google Scholar
- 25.Federal Bureau of Investigation. (1999). The school shooter: a threat assessment perspective. (Federal Bureau of Investigation, ED446352). Quantico VA. Retrieved from http://www.fbi.gov/library/school/school2.pdf.
- 26.Lingren T, Deleger L, Molnar K, Zhai H, Meinzen-Derr J, Kaiser M, et al. Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. J Am Med Inform Assoc. 2014;2013 https://doi.org/10.1136/amiajnl-2013-001837.CrossRefPubMedCentralGoogle Scholar
- 30.Zrimec, T., & Kononenko, I. (2004). Feasibility analysis of machine learning in medical diagnosis from aura images. In Proc. Int. Conf. KIRLIONICS-98 (Abstracts) (pp. 10–11).Google Scholar
- 31.Sara, NB, Halland R, Igel C, Alstrup S. High-school dropout prediction using machine learning: a Danish large-scale study European symposium on artificial neural networks, computational intelligence and machine learning. Bruges (Belgium), 22–24 April 2015.Google Scholar
- 32.Welsh JL, Schmidt F, McKinnon L, Chattha HK, Meyers JR A comparative study of adolescent risk assessment instruments: predictive and incremental validity assessment. 2008 Mar;15(1):104–15.Google Scholar
- 38.Neuman Y, Assaf D, Cohen Y, Knoll J. Profiling school shooters: automatic text-based analysis. Front Psych. 2015;6:1–5.Google Scholar