Journal of Quantitative Criminology

, Volume 12, Issue 1, pp 83–111 | Cite as

The effects of base rate and cutoff point choice on commonly used measures of association and accuracy in recidivism research

  • William R. Smith
Article

Abstract

Several measures of association and accuracy commonly used in the recidivism literature are examined for their sensitivity to variations in the proportion observed to recidivate (the base rate) and in the proportion selected for classification as recidivists (selection ratio). Logistic regression models are employed on a sample of 11,749 convicted offenders to generate predicted probabilities of four recidivism criteria varying in base rates from 0.06 to 0.48. Cutoff point selections from 0.1 to 0.9 show the effects of cutoff point changes on the following commonly used measures of association and accuracy: RIOC (relative improvement over chance), MCR (mean cost rating), Φ, γ, PRE (proportion reduction in error), and percentage correct. While all these statistics vary across base rate and cutoff points, some vary more than others: RIOC varies across cutoff points more than MCR, MCR more than Φ, and Φ more than γ. Researchers comparing such statistics across studies need be wary of the dangers of ignoring such variation.

Key Words

recidivism selection decisions association accuracy measures relative improvement over chance (RIOC) Φ mean cost rating γ 2×2 table 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Blumstein, A., Cohen, J., Roth, J. A., and Visher, C. A. (eds.) (1986).Criminal Careers and “Career Criminals” Vol. I, National Academy of Sciences Press, Washington, DC.Google Scholar
  2. Blumstein, A., Farrington, D. P., and Moitra, S. (1985). Delinquency careers: Innocents, desisters, and persisters. In Tonry, M., and Morris, N. (eds.),Crime and Justice: An Annual Review of Research, University of Chicago Press, Chicago.Google Scholar
  3. Cohen, J., and Cohen, P., (1983).Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 2nd ed., Lawrence Erlbaum Associates, Hillsdale, NJ.Google Scholar
  4. Copas, J. B., and Tarling, R. (1986). Some methodological issues in making predictions. In Blumstein, A.,et al. (eds.),Criminal Careers and “Career Criminals”. Vol. II, National Academy of Sciences Press, Washington, DC.Google Scholar
  5. Davis, J. A. (1971).Elementary Survey Analysis, Prentice-Hall, Englewood Cliffs, NJ.Google Scholar
  6. Duncan, O. D., Ohlin, L. E., Reiss, A. J., Jr., and Stanton, H. R. (1952). Formal devices for making selection decisions.Am. J. Sociol. 58: 573–584.Google Scholar
  7. Farrington, D. P., and Loeber, R. (1989). Relative improvement over chance (RIOC) and Φ as measures of predictive efficiency and strength of association in 2×2 tables.J. Quant. Criminol. 5: 201–213.CrossRefGoogle Scholar
  8. Farrington, D. P., and Tarling, R. (eds.) (1985).Prediction in Criminology. State University of New York Press, Albany.Google Scholar
  9. Geerkin, M. R., and Hayes, H. D. (1993). Probation and parole: Public risk and the future of incarceration alternatives.Criminology 31: 549–564.Google Scholar
  10. Glaser, D. (1954). A reconsideration of some parole prediction, factors.Am. Sociol. Rev. 19: 335–341.Google Scholar
  11. Gottfredson, D. M., Wilkins, L. T., and Hoffman, P. B. (1978).Guidelines for Parole and Sentencing, Lexington Books, Lexington, MA.Google Scholar
  12. Gottfredson, S. D., and Gottfredson, D. M. (1986). Accuracy of prediction models. In Blumstein, A.et al. (eds.),Criminal Careers and “Career Criminals”, Vol., II, National Academy of Sciences Press, Washington, DC.Google Scholar
  13. Hildebrand, D. K., Laing, J. D., and Rosenthal, H. (1977).Analysis of Ordinal Data, Sage, Beverly Hills, CA.Google Scholar
  14. Hoffman, P. B. (1983). Screening for risk: A revised Salient Factor Score (SFS 81).J. Crim. Just. 11: 539–547.Google Scholar
  15. Inciardi, J. A., Babst, D. V., and Koval, M. (1973). Computing mean cost ratings (MCR).J. Res. Crime Delinq. 10: 22–29.Google Scholar
  16. Kim, J. (1971). Predictive measures of ordinal association.Am. J. Sociol. 76: 891–907.CrossRefGoogle Scholar
  17. Lancucki, L., and Tarling, R. (1978). Appendix C: The relationship between mean cost rating and Kendall's rank correlation coefficient. In Gottfredson D. M., Wilkins, L., and Hoffman, P. B. (eds.)Guidelines for Parole and Sentencing, Heath, Lexington, MA.Google Scholar
  18. Loeber, R., and Dishion, T. (1983). Early predictors of male delinquency: A review.Psychol. Bull. 94: 68–99.Google Scholar
  19. Mosteller, F., and Tukey, J. W. (1977).Data Analysis and Regression. Addison-Wesley, Reading, MA.Google Scholar
  20. Mueller, J. H., Schuessler, K. F., and Costner, H. L. (1977).Statistical Reasoning in Sociology, 3rd ed., Houghton Mifflin Company, Boston.Google Scholar
  21. National Institute of Justice (1992).Data Resources of the National Institute of Justice, 5th ed., National Institute of Justice, Washington, DC.Google Scholar
  22. Petersilia, J., and Turner, S. (1987). Guideline-based justice: Prediction and racial minorities. In Gottfredson, D. M., and Tonry, M. (eds.),Prediction and Classification: Criminal Justice Decision Making, University of Chicago Press, Chicago.Google Scholar
  23. Reynolds, H. T. (1977).Analysis of Nominal Data, Sage, Beverly Hills, CA.Google Scholar
  24. Rhodes, W., Tyson, H., Weekley, J., Conly, C., and Powell, G. (1982).Developing Criteria for Identifying Career Criminals, U.S. Department of Justice, Washington, DC.Google Scholar
  25. Tarling, R. (1982). Comparison of measures of predictive power.Educ. Psychol. Measure. 42: 479–487.Google Scholar
  26. Visher, C. A. (1986). The Rand inmate survey: A reanalysis. In Blumstein, A.et al. (ed.),Criminal Careers and “Career Criminals”, Vol. II, National Academy of Sciences Press, Washington, DC.Google Scholar
  27. Visher, C. A., Lattimore, P. K., and Linster, R. L. (1991). Predicting the recidivism of serious youthful offenders using survival models.Criminology 29:329–366.CrossRefGoogle Scholar
  28. Wilbanks, W. L. (1985). Predicting failure on parole. In Farrington, D. P., and Tarling, R. (eds.),Prediction in Criminology, State University of New York Press, Albany.Google Scholar

Copyright information

© Plenum Publishing Corporation 1996

Authors and Affiliations

  • William R. Smith
    • 1
  1. 1.Department of Sociology and AnthropologyNorth Carolina State UniversityRaleigh

Personalised recommendations