Advertisement

Journal of Quantitative Criminology

, Volume 27, Issue 4, pp 547–573 | Cite as

A Comparison of Logistic Regression, Classification and Regression Tree, and Neural Networks Models in Predicting Violent Re-Offending

  • Yuan Y. Liu
  • Min YangEmail author
  • Malcolm Ramsay
  • Xiao S. LiEmail author
  • Jeremy W. Coid
Original Paper

Abstract

Previous studies that have compared logistic regression (LR), classification and regression tree (CART), and neural networks (NNs) models for their predictive validity have shown inconsistent results in demonstrating superiority of any one model. The three models were tested in a prospective sample of 1225 UK male prisoners followed up for a mean of 3.31 years after release. Items in a widely-used risk assessment instrument (the Historical, Clinical, Risk Management-20, or HCR-20) were used as predictors and violent reconvictions as outcome. Multi-validation procedure was used to reduce sampling error in reporting the predictive accuracy. The low base rate was controlled by using different measures in the three models to minimize prediction error and achieve a more balanced classification. Overall accuracy of the three models varied between 0.59 and 0.67, with an overall AUC range of 0.65–0.72. Although the performance of NNs was slightly better than that of LR and CART models, it did not demonstrate a significant improvement.

Keywords

Violence reconviction Risk assessment Neural networks Classification and regression tree HCR-20 

Notes

Acknowledgments

The project was funded by Ministry of Justice (England and Wales) and a grant from China Scholarship Council. Professor Min Yang and Professor Jeremy Coid were funded from the National Institute of Health Research Programme Grant (RP-PG-0407-10500). Malcolm Ramsay works for the Ministry of Justice. His contribution here is made in a personal capacity.

Conflict of interest

None.

References

  1. SPSS Inc. (2008) SPSS for windows, Rel. 16.0.1. 2008. SPSS Inc., Chicago. http://www.spss.com/
  2. StatSoft Inc. (2008) STATISTICA (data analysis software system), version 8.0. http://www.statsoft.com
  3. Banks S, Robbins PC, Silver E, Vesselinov R, Steadman HJ, Monahan J (2004) A multiple-models approach to violence risk assessment among people with mental disorder. Crim Justice Behav 31:324–340CrossRefGoogle Scholar
  4. Bigi R, Gregori D, Cortigiani L, Desideri A, Chiarotto FA, Toffolo GM (2005) Artificial neural networks and robust Bayesian classifiers for risk stratification following uncomplicated myocardial infarction. Int J Cardiol 101:481–487CrossRefGoogle Scholar
  5. Bishop C (1995) Neural networks for pattern recognition. Oxford University Press, New YorkGoogle Scholar
  6. Breiman L (2001) Decision tree forests. Mach Learn 45:5–32CrossRefGoogle Scholar
  7. Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Wadsworth and Brooks/Cole, Monterey, CAGoogle Scholar
  8. Brodzinski JD, Crable EA, Scherer RF (1994) Using artificial intelligence to model juvenile recidivism patterns. Comput Hum Serv 10:1–18Google Scholar
  9. Caulkins J, Cohen J, Gorr W, Wei J (1996) Predicting criminal recidivism: a comparison of neural network models with statistical methods. J Crim Just 24:227–240CrossRefGoogle Scholar
  10. Cicchetti DV (1992) Neural network and diagnosis in the clinical laboratory: state of the art. Clin Chem 38:9–10Google Scholar
  11. Cohen J (1990) Things I have learned (so far). Am Psychol 45:1304–1312CrossRefGoogle Scholar
  12. Coid JW, Yang M, Ullrich S, Zhang TQ, Sizmur S, Farrington DP (2010) Improving accuracy of risk prediction for violence: does changing the outcome matter? Int J Offender Ther (Submitted)Google Scholar
  13. Coid JW, Yang M, Ullrich S, Zhang TQ, Sizmur S, Roberts C, Farrington DP (2011) Most items in structured risk assessment instruments do not predict violence. J Forensic Psychiatr Psychol 22. doi: 10.1080/14789949.2010.495990
  14. Colombet I, Ruelland A, Chatellier G, Gueyffier F, Degoulet P, Jaulent MC (2000) Models to predict cardiovascular risk: comparison of CART, multilayer perceptron and logistic regression. In: Proceedings/AMIA annual symposium. AMIA symposium 2000, pp 156–160Google Scholar
  15. Comp.ai.neural-nets FAQ, Part 3: Generalization. http://www.faqs.org/faqs/ai-faq/neural-nets/part3/preamble.html
  16. Cooke DJ, Michie C, Ryan J (2001) Evaluating risk for violence: a preliminary study of the HCR-20, PCL-R and VRAG in a Scottish prison sample. Scottish Prison Service occasional papersGoogle Scholar
  17. Dahle KP (2006) Strengths and limitations of actuarial prediction of criminal reoffence in a German prison sample: a comparative study of LSI-R, HCR-20 and PCL-R. Int J Law Psychiat 29:431–442CrossRefGoogle Scholar
  18. Dawes RM, Faust D, Meehl PE (1989) Clinical versus actuarial judgement. Science 243:1668–1674CrossRefGoogle Scholar
  19. Derogatis LR, Melisaratos N (1983) The brief symptom inventory: an introductory report. Psychol Med 13:595–605CrossRefGoogle Scholar
  20. Dillard E, Luchette FA, Sears BW, Norton J, Schermer CR, Reed RL (2007) Clinician vs mathematical statistical models: which is better at predicting an abnormal chest radiograph finding in injured patients? Am J Emerg Med 25:823–830CrossRefGoogle Scholar
  21. Dolan M, Khawaja A (2004) The HCR-20 and post-discharge outcome in male patients discharged from medium security in the UK. Aggress Behav 30:469–483CrossRefGoogle Scholar
  22. Douglas KS, Ogloff JRP, Nicholls TL, Grant I (1999) Assessing risk for violence among psychiatric patients: the HCR-20 violence risk assessment scheme and the psychopathy checklist: screening version. J Consult Clin Psych 67:917–930CrossRefGoogle Scholar
  23. Doyle M, Dolan M (2006) Predicting community violence from patients discharged from mental health services. Brit J Psychiat 189:520–526CrossRefGoogle Scholar
  24. Farrington DP, Jolliffe D, Johnstone L (2008) Assessing violence risk: a framework for practice. Institute of Criminology, Cambridge University, CambridgeGoogle Scholar
  25. Florio T, Einfeld S, Levy F (1994) Neural networks and psychiatry: candidate applications in clinical decision making. Aust NZ J Psychiat 28:651–666CrossRefGoogle Scholar
  26. Forthofer MS, Bryant CA (2000) Using audience-segmentation techniques to tailor health behavior change strategies. Am J Health Behav 24:36–43Google Scholar
  27. Friedman JH (1999a) Greedy function approximation: a gradient boosting machine. IMS 1999 Reitz LectureGoogle Scholar
  28. Friedman JH (1999b) Stochastic gradient boosting. Stanford University, StanfordGoogle Scholar
  29. Gardner W, Lidz CW, Mulvey EP, Shaw EC (1996) A comparison of actuarial methods for identifying repetitively violent patients with mental illnesses. Law Human Behav 20:35–48CrossRefGoogle Scholar
  30. Gendreau P, Goggin C, Smith P (2002) Is the PCL-R really the “unparalleled” measure of offender risk? A lesson in knowledge cumulation. Crim Justice Behav 29:397–426CrossRefGoogle Scholar
  31. Gigerenzer G, Todd PM, Group AR (1999) Simple heuristics that makes us smart. Oxford University Press, New YorkGoogle Scholar
  32. Glover A, Nicholson D, Hemmati T, Bernfeld G, Quinsey V (2002) A comparison of predictors of general and violent recidivism among high risk federal offenders. Crim Justice Behav 29:235–249CrossRefGoogle Scholar
  33. Gottfredson SD, Moriarty LJ (2006) Statistical risk assessment: old problems and new applications. Crime Delinquency 52:178–200CrossRefGoogle Scholar
  34. Grann M, Langstrom N (2007) Actuarial Assessment of Violence Risk: To Weigh or Not to Weigh? Crim Justice Behav 34:22–36CrossRefGoogle Scholar
  35. Gray NS, Hill C, McGleish A, Timmons D, MacCulloch MJ, Snowden RJ (2003) Prediction of violence and self-harm in mentally disordered offenders: a prospective study of the efficacy of HCR-20, PCL-R and psychiatric symptomology. J Consult Clin Psych 71:443–451CrossRefGoogle Scholar
  36. Gray NS, Taylor J, Snowden RJ (2008) Predicting violent reconvictions using the HCR-20. Brit J Psychiat 192:384–387CrossRefGoogle Scholar
  37. Green DM, Swets JA (1966) Signal detection theory and psychophysics. Wiley, New YorkGoogle Scholar
  38. Greene MA, Hoffman PB, Beck JL (1994) The mean cost rating (MCR) is Somers’ D: a methodological note. J Crim Justice 22:63–69CrossRefGoogle Scholar
  39. Grevatt M, Thomas-Peter B, Hughes G (2004) Violence, mental disorder and risk assessment: can structured clinical assessments predict the short-term risk of inpatient violence? J Forensic Psychiat Psychol 15:278–292CrossRefGoogle Scholar
  40. Grove WM, Meehl PE (1996) Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychol Public Pol L 2:293–323CrossRefGoogle Scholar
  41. Guerriere MR, Detsky AS (1991) Neural networks: what are they? Ann Intern Med 115:906–907Google Scholar
  42. Hanson RK (2005) Twenty years of progress in violence risk assessment. J Interpers Violence 20:212–217CrossRefGoogle Scholar
  43. Harper PR (2005) A review and comparison of classification algorithms for medical decision making. Health Policy 71:315–331CrossRefGoogle Scholar
  44. Hart SD, Webster CD, Menzies RJ (1993) A note on portraying the accuracy of violence predictions. Law Human Behav 17:695–700CrossRefGoogle Scholar
  45. Hartvig P, Alfarnes S, Ostberg B, Skjønberg M, Moger TA (2006) Brief checklists for assessing violence risk among patients discharged from acute psychiatric facilities: a preliminary study. Nord J Psychiat 60:243–248CrossRefGoogle Scholar
  46. Hemphill JF, Hare RD, Wong S (1998) Psychopathy and recidivism: a review. Legal Criminol Psychol 3:139–170CrossRefGoogle Scholar
  47. Henderson AR (1993) Assessing test accuracy and its clinical consequences: a primer for receiver operating characteristic curve analysis. Ann Clin Biochem 30:521–539Google Scholar
  48. Hosmer DW, Lemeshow S (1989) Applied logistic regression. Wiley, New YorkGoogle Scholar
  49. Howard P, Kershaw C (2000) Using criminal career data in evaluation. British criminology conference: selected proceedings, 3. Available online at www.lboro.ac.uk/departments/ss/bsc/bccsp/vol03/howard.html
  50. Kass GV (1980) An exploratory technique for investigating large quantities of categorical data. Appl Stat 29:119–127CrossRefGoogle Scholar
  51. Kroner DG, Mills JF (2001) The accuracy of five risk appraisal instruments in predicting institutional misconduct and new convictions. Crim Justice Behav 28:471–489CrossRefGoogle Scholar
  52. Kroner DG, Mills JF, Reddon JR (2005) A coffee can, factor analysis, and prediction of antisocial behavior: the structure of criminal risk. Int J Law Psychiat 28:360–374CrossRefGoogle Scholar
  53. Lemon SC, Roy J, Clark MA, Friedmann PD, Rakowski W (2003) Classification and regression tree analysis in public health: methodological review and comparison with logistic regression. Ann Behav Med 26:172–181CrossRefGoogle Scholar
  54. Lidz CW, Mulvey EP, Gardner W (1993) The accuracy of predictions of violence to others. J Am Med Assn 269:1007–1011CrossRefGoogle Scholar
  55. Lin CH, Chou LS, Lin CH, Hsu CY, Chen YS, Lane HY (2007) Early prediction of clinical response in schizophrenia patients receiving the atypical antipsychotic zotepine. J Clin Psychiat 68:1522–1527CrossRefGoogle Scholar
  56. Loh WY, Shih YS (1997) Split selection methods for classification trees. Stat Sinica 7:815–840Google Scholar
  57. Manly BFJ (2005) Multivariate statistical methods: a primer, 3rd edn. Chapman and Hall/CRC, Boca RatonGoogle Scholar
  58. Maria-Pia Victoria-Feser (2000) Robust logistic regression for binomial responses. University of Geneva, GenevaGoogle Scholar
  59. McCulloch WS, Pitts W (1943) A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 5:115–133CrossRefGoogle Scholar
  60. Meehl PE, Rosen A (1955) Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychol Bull 52:194–216CrossRefGoogle Scholar
  61. Monahan J, Steadman HJ, Appelbaum PS, Robbins PC, Mulvey EP, Silver E (2000) Developing a clinically useful actuarial tool for assessing violence risk. Brit J Psychiat 176:312–320CrossRefGoogle Scholar
  62. Monahan J, Steadman HJ, Robbins PC, Appelbaum P, Banks S, Grisso T (2005) An actuarial model of violence risk assessment for persons with mental disorders. Psychiatr Serv 56:810–815CrossRefGoogle Scholar
  63. Monahan J, Steadman HJ, Appelbaum PS, Grisso T, Mulvey EP, Roth LH (2006) The classification of violence risk. Behav Sci Law 24:721–730CrossRefGoogle Scholar
  64. Mossman D (1994) Assessing predictions of violence: being accurate about accuracy. J Consult Clin Psych 62:783–792CrossRefGoogle Scholar
  65. National Institute of Justice (1992) Data resources of the National Institute of Justice, 5th edn. National Institute of Justice, Washington, DCGoogle Scholar
  66. Nicholls TL, Ogloff JRP, Douglas KS (2004) Assessing risk for violence among male and female civil psychiatric patients: the HCR-20, PCL: SV, and VSC. Behav Sci Law 22:127–158CrossRefGoogle Scholar
  67. Ning GM, Su J, Li YQ, Wang XY, Li CH, Yan WM (2006) Artificial neural network based model for cardiovascular risk stratification in hypertension. Med Biol Eng Comput 44:202–208CrossRefGoogle Scholar
  68. Palocsay SW, Wang P, Brookshire RG (2000) Predicting criminal recidivism using neural networks. Socio Econ Plan Sci 34:271–284CrossRefGoogle Scholar
  69. Patterson D (1996) Artificial neural networks. Prentice Hall, SingaporeGoogle Scholar
  70. Price RK, Spitznagel EL, Downey TJ, Meyer DJ, Risk NK, El-Ghazzawy OG (2000) Applying artificial neural network models to clinical decision making. Psychol Assess 12:40–51CrossRefGoogle Scholar
  71. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers, San MateoGoogle Scholar
  72. Quinlan JR (1996) Improved use of continuous attributes in C4.5. J Artif Intell Res 4:77–90Google Scholar
  73. Reiss AJ (1951) The accuracy, efficiency, and validity of a prediction instrument. Am J Sociol 61:552–561CrossRefGoogle Scholar
  74. Rice ME, Harris GT (1995a) Violent recidivism: assessing predictive validity. J Consult Clin Psych 63:737–748CrossRefGoogle Scholar
  75. Rice ME, Harris GT (1995b) Comparing effect sizes in follow-up studies: ROC area, Cohen’s d and r. Law Human Behav 29:615–620CrossRefGoogle Scholar
  76. Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, CambridgeGoogle Scholar
  77. Rosenfeld B, Harmon R (2002) Factors associated with violence in stalking and obsessional harassment cases. Crim Justice Behav 29:671–691CrossRefGoogle Scholar
  78. Rosenfeld B, Lewis C (2005) Assessing violence risk in stalking cases: a regression tree approach. Law Human Behav 29:343–357CrossRefGoogle Scholar
  79. Rumelhart DE, McClelland J (1986) Parallel distributed processing, vol 1. MIT Press, Cambridge, MAGoogle Scholar
  80. Shepherd AJ (1997) Second-order methods for neural networks. Springer, New YorkCrossRefGoogle Scholar
  81. Silver E, Chow-Martin L (2002) A multiple-models approach to assessing recidivism risk: implications for judicial decision making. Crim Justice Behav 29:538–568CrossRefGoogle Scholar
  82. Silver E, Smith WR, Banks S (2000) Constructing actuarial devices for predicting recidivism: a comparison of methods. Crim Justice Behav 27:733–764CrossRefGoogle Scholar
  83. Sjöstedt G, Grann M (2002) Risk assessment: what is being predicted by actuarial “prediction instruments”? Int J Forensic Ment Health 1:179–183Google Scholar
  84. Smith WR (1996) The effects of base rate and cutoff point choice on commonly used measures of association and accuracy in recidivism research. J Quant Criminol 12:83–111CrossRefGoogle Scholar
  85. Smith WR, Smith DR (1998) The consequences of error: recidivism prediction and civil-libertarian ratios. J Crim Just 26:481–502CrossRefGoogle Scholar
  86. Stalens LJ, Yarnold PR, Seng M, Olson DE, Repp M (2004) Identifying three types of violent offenders and predictinv violent recidivism while on probation: A classification tree analysis. Law Human Behav 28:253–271CrossRefGoogle Scholar
  87. Starzomska M (2003) Use of artificial neural networks in clinical psychology and psychiatry. Psychiat Polska 37:349–357Google Scholar
  88. StatSoft (2008) Data mining, predictive analytics, statistics, StatSoft electronic textbook. http://www.statsoft.com/textbook/
  89. Steadman HJ, Monahan J (1994) Toward a rejuvenation of risk assessment research. In: Monahan J, Steadman HJ (eds) Violence and mental disorder. University of Chicago Press, Chicago, pp 10–16Google Scholar
  90. Steadman HJ, Mulvey E, Monahan J, Robbins P, Appelbaum P, Grisso T (1998) Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Arch Gen Psychiat 55:393–401CrossRefGoogle Scholar
  91. Steadman HJ, Silver E, Monahan J, Appelbaum PS, Robbins PC, Mulvey EP (2000) A classification tree approach to the development of actuarial violence risk assessment tools. Law Human Behav 24:83–100CrossRefGoogle Scholar
  92. Tam KY, Kiang MY (1992) Managerial applications of neural networks: the case of bank failure predictions. Manage Sci 20:879–888Google Scholar
  93. Thomas S, Leese M (2003) A green-fingered approach can improve the clinical utility of violence risk assessment tools. Crim Behav Ment Health 13:153–158CrossRefGoogle Scholar
  94. Thomas S, Leesea M, Walsh E, McCrone P, Moran P, Burns T (2005) A comparison of statistical models in predicting violence in psychotic illness. Compr Psychiat 46:296–303CrossRefGoogle Scholar
  95. Trujillano J, Sarria-Santamera A, Esquerda A, Badia M, Palma M, March J (2008) Approach to the methodology of classification and regression trees. Gaceta Sanitaria/SESPAS 22:65–72CrossRefGoogle Scholar
  96. Tu JV (1996) Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 9:1225–1231CrossRefGoogle Scholar
  97. UK700 Group (1999) Predictors of quality of life in people with severe mental illness. Study methodology with baseline analysis in the UK700 trial. Brit J Psychiat 175:426–432CrossRefGoogle Scholar
  98. Vogel V, Ruiter C, Hildebrand M, Bos B, Ven P (2004) Type of discharge and risk of recidivism measured by the HCR-20: a retrospective study in a Dutch sample of treated forensic psychiatric patients. Int J Forensic Ment Health 3:149–165Google Scholar
  99. Webster CD, Douglas KS, Eaves D, Hart S (1997) HCR-20: assessing risk for violence (version 2). Simon Fraser University, Vancouver, CanadaGoogle Scholar
  100. Webster CD, Muller-Isberner R, Frannson G (2002) Violence risk assessment: using structured clinical guidelines professionally. Int J Ment Health 2:185–193Google Scholar
  101. Yang M, Liu YY, Coid JW (2010) Applying neural networks and classification tree models to the classification of serious offenders and the prediction of recidivism. Research Summary, Ministry of Justice, UK. Available online at www.justice.gov.uk/publications/research.htm
  102. Yarnold PR (1996) Discriminating geriatric and nongeriatric patients using functional status information: an example of classification tree analysis via UniODA. Educ Psychol Meas 56:656–667CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Health StatisticsSchool of Public Health, Sichuan UniversityChengduChina
  2. 2.Division of PsychiatrySchool for Community Health Sciences, University of NottinghamNottinghamUK
  3. 3.Partnerships and Health Strategy Unit, Ministry of JusticeLondonUK
  4. 4.Forensic Psychiatry Research Unit, Queen Mary University of LondonLondonUK

Personalised recommendations