Transparency, Accuracy and Fairness

  • Richard Berk


Criminal justice risk assessments are often far more than academic exercises. They can serve as informational input to a range of real decisions affecting real people. The consequences of these decisions can be enormous, and they can be made in error. Stakeholders need to know about the risk assessment tools being deployed. The need to know includes transparency, accuracy, and fairness. All three raise complicated issues in part because they interact with one another. Each will be addressed in turn. There will be no technical fix and no easy answers.


  1. Arrow, K. (1950) A difficulty in the concept of social welfare. Journal of Political Economy 58(4) 328–346.CrossRefGoogle Scholar
  2. Berk, R. A., Heirdari, H., Jabbari, S., Kearns, M., & Roth, A. (2018a) Fairness in criminal justice risk assessments: The State of the Art. Sociological Methods and Research, in press.Google Scholar
  3. Chouldechova, A. (2017) Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. arXiv:1703.00056v1 [stat. AP].CrossRefGoogle Scholar
  4. Coglianese, C., & Lehr, D. (2018a) Transparency and algorithmic governance. Working Paper. Penn Program on Regulation, University of Pennsylvania Law School.Google Scholar
  5. Coglianese, C., & Lehr, D. (2018b) Algorithm vs. algorithm: placing regulatory use of machine learning in perspective. Working Paper. Penn Program on Regulation, University of Pennsylvania Law School.Google Scholar
  6. Corbett-Davies, S. & Goel, S. (2018) The measure and mismeasure of fairness: a critical review of fair machine learning. 35th International Conference on Machine Learning (ICML 2018).Google Scholar
  7. Dana, J., & Dawes, R. M. (2004). The superiority of simple alternatives to regression for social science predictions. Journal of Educational and Behavioral Statistics 29(3): 317–331.CrossRefGoogle Scholar
  8. Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science 243(4899): 1668–1674.CrossRefGoogle Scholar
  9. Hand, D.J. (2009). Measuring classifier performance: A coherent alternative to the area under the ROC curve. Machine Learning 77: 103–123.CrossRefGoogle Scholar
  10. Huq, A.Z. (2019) Racial equality in algorithmic criminal justice. Duke Law Journal 68, forthcoming.Google Scholar
  11. Kearns, M., Neel, S., Roth, A, & Wu, Z. (2018a) Preventing fairness gerrymandering: auditing and learning subgroup fairness. arXiv:1711.05144v4 [cs.LG].Google Scholar
  12. Kearns, M., Neel, S., Roth, A, & Wu, Z. (2018b) An empirical study of rich subgroup fairness for machine learning. asXiv:1808.08166v1 [cs.LG]Google Scholar
  13. Kleinman, M., Ostrom, B. J., & Cheeman, F. L. (2007) Using risk assessment to inform sentencing decisions for nonviolent offenders in Virginia. Crime & Delinquency 53(1): 1–27.Google Scholar
  14. Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017b) Inherent tradeoffs in the fair determination of risk scores. Proc. 8th Conference on Innovations in Theoretical Computer Science (ITCS).Google Scholar
  15. Kroll, J.A., Huey, J., Barocas, S., Felten, E.W., Reidenberg, J.R., Robinson, D.G., & Yu, H. (2018) Accountable algorithms. University of Pennsylvania Law Review 165: 633–705.Google Scholar
  16. Lobo, J. M.; Jiménez-Valverde, A., & Real, R. (2008). AUC: a misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography 17: 145–151.CrossRefGoogle Scholar
  17. Mease, D., Wyner, A.J., & Buja, A. (2007) Boosted classification trees and class probability/quantile estimation. Journal of Machine Learning Research 8: 409–439.zbMATHGoogle Scholar
  18. Sen, A. (2018) Collective Choice and Social Welfare Cambridge: Harvard university PresszbMATHGoogle Scholar
  19. Zeng, J., Ustan, B., & Rudin, C. (2017) Interpretable classification models for recidivism prediction. Journal of the Royal Statistical Society: Series A 180(3): 689–722.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Richard Berk
    • 1
  1. 1.Department of CriminologyUniversity of PennsylvaniaPhiladelphiaUSA

Personalised recommendations