A Causal Bayesian Networks Viewpoint on Fairness

  • Silvia ChiappaEmail author
  • William S. Isaac
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 547)


We offer a graphical interpretation of unfairness in a dataset as the presence of an unfair causal effect of the sensitive attribute in the causal Bayesian network representing the data-generation mechanism. We use this viewpoint to revisit the recent debate surrounding the COMPAS pretrial risk assessment tool and, more generally, to point out that fairness evaluation on a model requires careful considerations on the patterns of unfairness underlying the training data. We show that causal Bayesian networks provide us with a powerful tool to measure unfairness in a dataset and to design fair models in complex unfairness scenarios.



The authors would like to thank Ray Jiang, Christina Heinze-Deml, Tom Stepleton, Tom Everitt, and Shira Mitchell for useful discussions.


  1. 1.
    AI Now Institute. Litigating Algorithms: Challenging Government Use of Algorithmic Decision Systems (2018)Google Scholar
  2. 2.
    Alexander, M.: The New Jim Crow: Mass Incarceration in the Age of Colorblindness. The New Press, New York (2012)Google Scholar
  3. 3.
    Andrews, D.A., Bonta, J.: Level of Service Inventory - Revised. Multi-Health Systems, Toronto (2000)Google Scholar
  4. 4.
    Andrews, D.A., Bonta, J., Wormith, J.S.: The recent past and near future of risk and/or need assessment. Crime Delinq. 52(1), 7–27 (2006)CrossRefGoogle Scholar
  5. 5.
    Angwin, J., Larson, J., Mattu, S., Kirchner, L.: Machine Bias: there’s software used across the country to predict future criminals. And it’s biased against blacks, May 2016.
  6. 6.
    Arnold, D., Dobbie, W., Yang, C.S.: Racial bias in bail decisions. Q. J. Econ. 133, 1885–1932 (2018)CrossRefGoogle Scholar
  7. 7.
    Berk, R., Heidari, H., Jabbari, S., Kearns, M., Roth, A.: Fairness in criminal justice AQ3 risk assessments: the state of the art. Sociol. Methods Res. (2018)Google Scholar
  8. 8.
    Bogen, M., Rieke, A.: Help wanted: an examination of hiring algorithms, equity, and bias. Technical report, Upturn (2018)Google Scholar
  9. 9.
    Brennan, T., Dieterich, W., Ehret, B.: Evaluating the predictive validity of the COMPAS risk and needs assessment system. Crim. Justice Behav. 36(1), 21–40 (2009)CrossRefGoogle Scholar
  10. 10.
    Byanjankar, A., Heikkilä, M., Mezei, J.: Predicting credit risk in peer-to-peer lending: a neural network approach. In: IEEE Symposium Series on Computational Intelligence, pp. 719–725 (2015)Google Scholar
  11. 11.
    Chiappa, S.: Path-specific counterfactual fairness. In: Thirty-Third AAAI Conference on Artificial Intelligence (2019)Google Scholar
  12. 12.
    Chouldechova, A.: Fair prediction with disparate impact: a study of bias in recidivism prediction instruments. Big Data 5(2), 153–163 (2017)CrossRefGoogle Scholar
  13. 13.
    Chouldechova, A., Putnam-Hornstein, E., Benavides-Prado, D., Fialko, O., Vaithianathan, R.: A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. Proc. Mach. Learn. Res. 81, 134–148 (2018)Google Scholar
  14. 14.
    Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear, October 2016.
  15. 15.
    Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 797–806 (2017)Google Scholar
  16. 16.
    Corbett-Davies, S., Goel, S.: The measure and mismeasure of fairness: a critical review of fair machine learning. CoRR, abs/1808.00023 (2018)Google Scholar
  17. 17.
    Dawid, P.: Fundamentals of statistical causality. Technical report, University College London (2007)Google Scholar
  18. 18.
    De Fauw, J., et al.: Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat. Med. 24(9), 1342–1350 (2018)CrossRefGoogle Scholar
  19. 19.
    Dieterich, W., Mendoza, C., Brennan, T.: COMPAS risk scales: demonstrating accuracy equity and predictive parity (2016)Google Scholar
  20. 20.
    Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226 (2012)Google Scholar
  21. 21.
    Eckhouse, L., Lum, K., Conti-Cook, C., Ciccolini, J.: Layers of bias: a unified approach for understanding problems with risk assessment. Crim. Justice Behav. 46, 185–209 (2018)CrossRefGoogle Scholar
  22. 22.
    Eubanks, V.: Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin’s Press, New York (2018)Google Scholar
  23. 23.
    Feldman, M., Friedler, S.A., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 259–268 (2015)Google Scholar
  24. 24.
    Flores, A.W., Bechtel, K., Lowenkamp, C.T.: False positives, false negatives, and false analyses: a rejoinder to “Machine Bias: there’s software used across the country to predict future criminals. And it’s biased against blacks”. Fed. Probat. 80(2), 38–46 (2016)Google Scholar
  25. 25.
    Harvard Law School. Note: Bail reform and risk assessment: The cautionary tale of federal sentencing. Harvard Law Rev. 131(4), 1125–1146 (2018)Google Scholar
  26. 26.
    He, X., et al.: Practical lessons from predicting clicks on ads at Facebook. In: Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, pp. 1–9 (2014)Google Scholar
  27. 27.
    Hoffman, M., Kahn, L.B., Li, D.: Discretion in hiring. Q. J. Econ. 133(2), 765–800 (2018)CrossRefGoogle Scholar
  28. 28.
    Isaac, W.S.: Hope, hype, and fear: the promise and potential pitfalls of artificial intelligence in criminal justice. Ohio State J. Crim. Law 15(2), 543–558 (2017)MathSciNetGoogle Scholar
  29. 29.
    Kleinberg, J., Mullainathan, S., Raghavan, M.: Inherent trade-offs in the fair determination of risk scores. In: 8th Innovations in Theoretical Computer Science Conference, pp. 43:1–43:23 (2016)Google Scholar
  30. 30.
    Koepke, J.L., Robinson, D.G.: Danger ahead: risk assessment and the future of bail reform. Wash. Law Rev. 93, 1725–1807 (2017)Google Scholar
  31. 31.
    Kourou, K., Exarchos, T.P., Exarchos, K.P., Karamouzis, M.V., Fotiadis, D.I.: Machine learning applications in cancer prognosis and prediction. Comput. Struct. Biotechnol. J. 13, 8–17 (2015)CrossRefGoogle Scholar
  32. 32.
    Kusner, M.J., Loftus, J.R., Russell, C., Silva, R.: Counterfactual fairness. In: Advances in Neural Information Processing Systems 30, pp. 4069–4079 (2017)Google Scholar
  33. 33.
    Lum, K.: Limitations of mitigating judicial bias with machine learning. Nat. Hum. Behav. 1(7), 1 (2017)CrossRefGoogle Scholar
  34. 34.
    Lum, K., Isaac, W.: To predict and serve? Significance 13(5), 14–19 (2016)CrossRefGoogle Scholar
  35. 35.
    Malekipirbazari, M., Aksakalli, V.: Risk assessment in social lending via random forests. Expert Syst. Appl. 42(10), 4621–4631 (2015)CrossRefGoogle Scholar
  36. 36.
    Mayson, S.G.: Bias in, bias out. Yale Law Sch. J. 128 (2019)Google Scholar
  37. 37.
    Mitchell, S., Potash, E., Barocas, S.: Prediction-based decisions and fairness: a catalogue of choices, assumptions, and definitions (2018)Google Scholar
  38. 38.
    Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  39. 39.
    Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. Wiley, Hoboken (2016)zbMATHGoogle Scholar
  40. 40.
    Perlich, C., Dalessandro, B., Raeder, T., Stitelman, O., Provost, F.: Machine learning for targeted display advertising: transfer learning in action. Mach. Learn. 95(1), 103–127 (2014)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Peters, J., Janzing, D., Schölkopf, B.: Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press, Cambridge (2017)zbMATHGoogle Scholar
  42. 42.
    Rosenberg, M., Levinson, R.: Trump’s catch-and-detain policy snares many who call the U.S. home, June 2018.
  43. 43.
    Selbst, A.D.: Disparate impact in big data policing. Georgia Law Rev. 52, 109–195 (2017)Google Scholar
  44. 44.
    Spirtes, P., et al.: Causation, Prediction, and Search. MIT Press, Cambridge (2000)zbMATHGoogle Scholar
  45. 45.
    Stevenson, M.T.: Assessing risk assessment in action. Minnesota Law Rev. 103, 303 (2017)Google Scholar
  46. 46.
    Vaithianathan, R., Maloney, T., Putnam-Hornstein, E., Jiang, N.: Children in the public benefit system at risk of maltreatment: identification via predictive modeling. Am. J. Prev. Med. 45(3), 354–359 (2013)CrossRefGoogle Scholar
  47. 47.
    Zhang, J., Bareinboim, E.: Fairness in decision-making - the causal explanation formula. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  1. 1.DeepMindLondonUK

Personalised recommendations