Abstract
Several measures of association and accuracy commonly used in the recidivism literature are examined for their sensitivity to variations in the proportion observed to recidivate (the base rate) and in the proportion selected for classification as recidivists (selection ratio). Logistic regression models are employed on a sample of 11,749 convicted offenders to generate predicted probabilities of four recidivism criteria varying in base rates from 0.06 to 0.48. Cutoff point selections from 0.1 to 0.9 show the effects of cutoff point changes on the following commonly used measures of association and accuracy: RIOC (relative improvement over chance), MCR (mean cost rating), Φ, γ, PRE (proportion reduction in error), and percentage correct. While all these statistics vary across base rate and cutoff points, some vary more than others: RIOC varies across cutoff points more than MCR, MCR more than Φ, and Φ more than γ. Researchers comparing such statistics across studies need be wary of the dangers of ignoring such variation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Blumstein, A., Cohen, J., Roth, J. A., and Visher, C. A. (eds.) (1986).Criminal Careers and “Career Criminals” Vol. I, National Academy of Sciences Press, Washington, DC.
Blumstein, A., Farrington, D. P., and Moitra, S. (1985). Delinquency careers: Innocents, desisters, and persisters. In Tonry, M., and Morris, N. (eds.),Crime and Justice: An Annual Review of Research, University of Chicago Press, Chicago.
Cohen, J., and Cohen, P., (1983).Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 2nd ed., Lawrence Erlbaum Associates, Hillsdale, NJ.
Copas, J. B., and Tarling, R. (1986). Some methodological issues in making predictions. In Blumstein, A.,et al. (eds.),Criminal Careers and “Career Criminals”. Vol. II, National Academy of Sciences Press, Washington, DC.
Davis, J. A. (1971).Elementary Survey Analysis, Prentice-Hall, Englewood Cliffs, NJ.
Duncan, O. D., Ohlin, L. E., Reiss, A. J., Jr., and Stanton, H. R. (1952). Formal devices for making selection decisions.Am. J. Sociol. 58: 573–584.
Farrington, D. P., and Loeber, R. (1989). Relative improvement over chance (RIOC) and Φ as measures of predictive efficiency and strength of association in 2×2 tables.J. Quant. Criminol. 5: 201–213.
Farrington, D. P., and Tarling, R. (eds.) (1985).Prediction in Criminology. State University of New York Press, Albany.
Geerkin, M. R., and Hayes, H. D. (1993). Probation and parole: Public risk and the future of incarceration alternatives.Criminology 31: 549–564.
Glaser, D. (1954). A reconsideration of some parole prediction, factors.Am. Sociol. Rev. 19: 335–341.
Gottfredson, D. M., Wilkins, L. T., and Hoffman, P. B. (1978).Guidelines for Parole and Sentencing, Lexington Books, Lexington, MA.
Gottfredson, S. D., and Gottfredson, D. M. (1986). Accuracy of prediction models. In Blumstein, A.et al. (eds.),Criminal Careers and “Career Criminals”, Vol., II, National Academy of Sciences Press, Washington, DC.
Hildebrand, D. K., Laing, J. D., and Rosenthal, H. (1977).Analysis of Ordinal Data, Sage, Beverly Hills, CA.
Hoffman, P. B. (1983). Screening for risk: A revised Salient Factor Score (SFS 81).J. Crim. Just. 11: 539–547.
Inciardi, J. A., Babst, D. V., and Koval, M. (1973). Computing mean cost ratings (MCR).J. Res. Crime Delinq. 10: 22–29.
Kim, J. (1971). Predictive measures of ordinal association.Am. J. Sociol. 76: 891–907.
Lancucki, L., and Tarling, R. (1978). Appendix C: The relationship between mean cost rating and Kendall's rank correlation coefficient. In Gottfredson D. M., Wilkins, L., and Hoffman, P. B. (eds.)Guidelines for Parole and Sentencing, Heath, Lexington, MA.
Loeber, R., and Dishion, T. (1983). Early predictors of male delinquency: A review.Psychol. Bull. 94: 68–99.
Mosteller, F., and Tukey, J. W. (1977).Data Analysis and Regression. Addison-Wesley, Reading, MA.
Mueller, J. H., Schuessler, K. F., and Costner, H. L. (1977).Statistical Reasoning in Sociology, 3rd ed., Houghton Mifflin Company, Boston.
National Institute of Justice (1992).Data Resources of the National Institute of Justice, 5th ed., National Institute of Justice, Washington, DC.
Petersilia, J., and Turner, S. (1987). Guideline-based justice: Prediction and racial minorities. In Gottfredson, D. M., and Tonry, M. (eds.),Prediction and Classification: Criminal Justice Decision Making, University of Chicago Press, Chicago.
Reynolds, H. T. (1977).Analysis of Nominal Data, Sage, Beverly Hills, CA.
Rhodes, W., Tyson, H., Weekley, J., Conly, C., and Powell, G. (1982).Developing Criteria for Identifying Career Criminals, U.S. Department of Justice, Washington, DC.
Tarling, R. (1982). Comparison of measures of predictive power.Educ. Psychol. Measure. 42: 479–487.
Visher, C. A. (1986). The Rand inmate survey: A reanalysis. In Blumstein, A.et al. (ed.),Criminal Careers and “Career Criminals”, Vol. II, National Academy of Sciences Press, Washington, DC.
Visher, C. A., Lattimore, P. K., and Linster, R. L. (1991). Predicting the recidivism of serious youthful offenders using survival models.Criminology 29:329–366.
Wilbanks, W. L. (1985). Predicting failure on parole. In Farrington, D. P., and Tarling, R. (eds.),Prediction in Criminology, State University of New York Press, Albany.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Smith, W.R. The effects of base rate and cutoff point choice on commonly used measures of association and accuracy in recidivism research. J Quant Criminol 12, 83–111 (1996). https://doi.org/10.1007/BF02354472
Issue Date:
DOI: https://doi.org/10.1007/BF02354472