Abstract
School violence has increased over the past decade and innovative, sensitive, and standardized approaches to assess school violence risk are needed. In our current feasibility study, we initialized a standardized, sensitive, and rapid school violence risk approach with manual annotation. Manual annotation is the process of analyzing a student’s transcribed interview to extract relevant information (e.g., key words) to school violence risk levels that are associated with students’ behaviors, attitudes, feelings, use of technology (social media and video games), and other activities. In this feasibility study, we first implemented school violence risk assessments to evaluate risk levels by interviewing the student and parent separately at the school or the hospital to complete our novel school safety scales. We completed 25 risk assessments, resulting in 25 transcribed interviews of 12–18 year olds from 15 schools in Ohio and Kentucky. We then analyzed structured professional judgments, language, and patterns associated with school violence risk levels by using manual annotation and statistical methodology. To analyze the student interviews, we initiated the development of an annotation guideline to extract key information that is associated with students’ behaviors, attitudes, feelings, use of technology and other activities. Statistical analysis was applied to associate the significant categories with students’ risk levels to identify key factors which will help with developing action steps to reduce risk. In a future study, we plan to recruit more subjects in order to fully develop the manual annotation which will result in a more standardized and sensitive approach to school violence assessments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Robers S, Kemp J, Truman J, Snyder TD: Indicators of School Crime and Safety: 2012. Washington: US Department of Education and US Department of Justice Office of Justice Programs, 2013.
National Association of School Psychologists. (2010). Crisis and safety resources. Retrieved April 3, 2014 from http://www.nasponline.org/educators/index.aspx#crisis.
Gottfredson GD, Cook PJ, NA C: Schools and Prevention. In: Welsh BC, Farrington DP (Eds): Crime and Prevention. Oxford, United Kingdom: Oxford University Press, pp. 269–287, 2000.
Tanner-Smith EE, Wilson SJ, Lipsey MW: Risk Factors and Crime. In: Maguire M, Morgan R, Reiner R (Eds) The Oxford Handbook of Criminology. 5th edn, Oxford, Oxford University Press, pp. 89–111, 2012.
Borum R, Cornell DG, Modzeleski W, Jimerson SR: What can be done about school shootings? A Review of the Evidence. Educational Researcher 39(1): 27–37, 2010.
Nekvasil EK, Cornell DG: Student reports of peer threats of violence: Prevalence and outcomes. Journal of School Violence 11(4): 357–375, 2012.
Bernes KB, Bardick AD: Conducting adolescent violence risk assessments: A framework for school counselors. Professional School Counseling 10(4): 419–427, 2007.
McGowan MR, Horn RA, Mellott RN: The predictive validity of the structured assessment of violence risk in youth in secondary educational settings. Psychological Assessment 23(2): 478–486, 2011.
Monahan J, Steadman H: Violence Risk Assessment: A Quarter Century of Research. In: Frost L, Bonnie R (Eds.): The Evolution of Mental Health Law. Washington: American Psychological Association, pp. 195–211, 2001. doi:10.1037/10414-010.
Barzman D, Brackenbury L, Sonnier L, Schnell B, Cassedy A, Salisbury S, Sorter M, Mossman D: Brief rating of aggression by children and adolescents (BRACHA): Development of a Tool to Assess Risk of Inpatients’ Aggressive Behavior. Journal of the American Academy of Psychiatry and the Law 39(2): 170–179, 2011.
South BR, Shen S, Jones M, Garvin J, Samore MH, Chapman WW, Gundlapalli AV: Developing a manually annotated clinical document corpus to identify phenotypic information for inflammatory bowel disease. BMC Bioinformatics 10(Suppl 9):S12, 2009.
Roberts A, Gaizauskas R, Hepple M, Demetriou G, Guo Y, Roberts I, Setzer A: Building a semantically annotated corpus of clinical texts. Journal of biomedical informatics 42(5):950–966, 2009.
Xia F, Yetisgen-Yildiz: Clinical corpus annotation: challenges and strategies. Proc. Of Third Workshop on Building and Evaluating Resources for Biomedical Text Mining of the International Conference on Language Resources and Evaluation, 2012.
Deleger L, Brodzinski H, Zhai H, Li Q, Lingren T, Kirkendall ES, Alessandrini E, Solti I: Developing and evaluating an automated appendicitis risk stratification algorithm for pediatric patients in the emergency department. Journal of the American Medical Informatics Association, 20(e2): e212–e220, 2013.
Li Q, Kirkendall E, Hall E, Ni Y, Lingren T, Kaiser M, Lingren N, Zhai H, Solti I, Melton K: Automated detection of medication and fluid administration errors in neonatal intensive care. Journal of Biomedical Informatics, 2015. doi:10.1016/j.jbi.2015.07.012.
Li Q, Spooner SA, Kaiser M, Lingren N, Robbins J, Lingren T, Tang H, Solti I, Ni Y: An end-to-end hybrid algorithm for automated medication discrepancy detection. BMC Medical Informatics and Decision Making 15(1):37, 2015.
Barzman D, Mossman D, Sonnier L, Sorter M: Brief rating of aggression by children and adolescents (BRACHA): A reliability study. Journal of the American Academy of Psychiatry and the Law 40:374–382, 2012.
Delgado SV, Barzman D, Gehle M, Caring M, Sorter MD, Kowatch R, Finding R: Characteristics of Discharges Against Medical Advice from Acute Inpatient Psychiatric Units for Children and Adolescents. Poster presented at the annual meeting of the American Academy of Child and Adolescent Psychiatry, Boston, 2007.
Douglas KS, Blanchard AJE, Guy LS, Reeves KA, Weir J (2010). HCR-20 Violence Risk Assessment Scheme: Overview and Annotated Bibliography.Retrieved from http://kdouglas.files.wordpress.com/2007/10/hcr-20-annotated-biblio-sept-2010.pdf.
Hilterman EL, Nicholls TL, van Nieuwenhuizen C: Predictive performance of Risk Assessments in Juvenile Offenders: Comparing the SAVRY, PCL:YV, and YLS/CMI With Unstructured Clinical Assessments. Assessment, 2014.
Federal Bureau of Investigation. (1999). The School Shooter: A Threat Assessment Perspective. (Federal Bureau of Investigation, ED446352). Quantico VA. Retrieved from http://www.fbi.gov/library/school/school2.pdf.
Lingren T, Deleger L, Molnar K, Zhai H, Meinzen-Derr J, Kaiser M, Stoutenborough L, Li Q, Solti I: Evaluating the impact of pre-annotation on annotation speed and potential bias: Natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. Journal of the American Medical Informatics Association, 2013. doi:10.1136/amiajnl-2013-001837.
Ogren P, Guergana S, Christopher C: Constructing evaluation corpora for automated clinical named entity recognition: In: Proc. of the sixth international conference on language resources and evaluation (LREC), 2008.
Welsh JL, Schmidt F, McKinnon L, Chattha HK, Meyers JR: A comparative study of adolescent risk assessment instruments: predictive and incremental validity. Assessment 15:104–115, 2008.
Falzer PR: Valuing structured professional judgment: Predictive performance, decision-making, and the clinical-actuarial conflict. Behavioral Sciences and the Law 31:40–54, 2013.
Acknowledgments
The Benderson Family and Cincinnati Children’s Hospital Medical Center.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
All authors declare that they have no conflict of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Rights and permissions
About this article
Cite this article
Barzman, D.H., Ni, Y., Griffey, M. et al. A Pilot Study on Developing a Standardized and Sensitive School Violence Risk Assessment with Manual Annotation. Psychiatr Q 88, 447–457 (2017). https://doi.org/10.1007/s11126-016-9458-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11126-016-9458-7