# A Causal Bayesian Networks Viewpoint on Fairness

## Abstract

We offer a graphical interpretation of unfairness in a dataset as the presence of an unfair causal effect of the sensitive attribute in the causal Bayesian network representing the data-generation mechanism. We use this viewpoint to revisit the recent debate surrounding the COMPAS pretrial risk assessment tool and, more generally, to point out that fairness evaluation on a model requires careful considerations on the patterns of unfairness underlying the training data. We show that causal Bayesian networks provide us with a powerful tool to measure unfairness in a dataset and to design fair models in complex unfairness scenarios.

## Notes

### Acknowledgements

The authors would like to thank Ray Jiang, Christina Heinze-Deml, Tom Stepleton, Tom Everitt, and Shira Mitchell for useful discussions.

## References

- 1.AI Now Institute. Litigating Algorithms: Challenging Government Use of Algorithmic Decision Systems (2018)Google Scholar
- 2.Alexander, M.: The New Jim Crow: Mass Incarceration in the Age of Colorblindness. The New Press, New York (2012)Google Scholar
- 3.Andrews, D.A., Bonta, J.: Level of Service Inventory - Revised. Multi-Health Systems, Toronto (2000)Google Scholar
- 4.Andrews, D.A., Bonta, J., Wormith, J.S.: The recent past and near future of risk and/or need assessment. Crime Delinq.
**52**(1), 7–27 (2006)CrossRefGoogle Scholar - 5.Angwin, J., Larson, J., Mattu, S., Kirchner, L.: Machine Bias: there’s software used across the country to predict future criminals. And it’s biased against blacks, May 2016. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- 6.Arnold, D., Dobbie, W., Yang, C.S.: Racial bias in bail decisions. Q. J. Econ.
**133**, 1885–1932 (2018)CrossRefGoogle Scholar - 7.Berk, R., Heidari, H., Jabbari, S., Kearns, M., Roth, A.: Fairness in criminal justice AQ3 risk assessments: the state of the art. Sociol. Methods Res. (2018)Google Scholar
- 8.Bogen, M., Rieke, A.: Help wanted: an examination of hiring algorithms, equity, and bias. Technical report, Upturn (2018)Google Scholar
- 9.Brennan, T., Dieterich, W., Ehret, B.: Evaluating the predictive validity of the COMPAS risk and needs assessment system. Crim. Justice Behav.
**36**(1), 21–40 (2009)CrossRefGoogle Scholar - 10.Byanjankar, A., Heikkilä, M., Mezei, J.: Predicting credit risk in peer-to-peer lending: a neural network approach. In: IEEE Symposium Series on Computational Intelligence, pp. 719–725 (2015)Google Scholar
- 11.Chiappa, S.: Path-specific counterfactual fairness. In: Thirty-Third AAAI Conference on Artificial Intelligence (2019)Google Scholar
- 12.Chouldechova, A.: Fair prediction with disparate impact: a study of bias in recidivism prediction instruments. Big Data
**5**(2), 153–163 (2017)CrossRefGoogle Scholar - 13.Chouldechova, A., Putnam-Hornstein, E., Benavides-Prado, D., Fialko, O., Vaithianathan, R.: A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. Proc. Mach. Learn. Res.
**81**, 134–148 (2018)Google Scholar - 14.Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear, October 2016. https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/?utm_term=.8c6e8c1cfbdf
- 15.Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 797–806 (2017)Google Scholar
- 16.Corbett-Davies, S., Goel, S.: The measure and mismeasure of fairness: a critical review of fair machine learning. CoRR, abs/1808.00023 (2018)Google Scholar
- 17.Dawid, P.: Fundamentals of statistical causality. Technical report, University College London (2007)Google Scholar
- 18.De Fauw, J., et al.: Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat. Med.
**24**(9), 1342–1350 (2018)CrossRefGoogle Scholar - 19.Dieterich, W., Mendoza, C., Brennan, T.: COMPAS risk scales: demonstrating accuracy equity and predictive parity (2016)Google Scholar
- 20.Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226 (2012)Google Scholar
- 21.Eckhouse, L., Lum, K., Conti-Cook, C., Ciccolini, J.: Layers of bias: a unified approach for understanding problems with risk assessment. Crim. Justice Behav.
**46**, 185–209 (2018)CrossRefGoogle Scholar - 22.Eubanks, V.: Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin’s Press, New York (2018)Google Scholar
- 23.Feldman, M., Friedler, S.A., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 259–268 (2015)Google Scholar
- 24.Flores, A.W., Bechtel, K., Lowenkamp, C.T.: False positives, false negatives, and false analyses: a rejoinder to “Machine Bias: there’s software used across the country to predict future criminals. And it’s biased against blacks”. Fed. Probat.
**80**(2), 38–46 (2016)Google Scholar - 25.Harvard Law School. Note: Bail reform and risk assessment: The cautionary tale of federal sentencing. Harvard Law Rev.
**131**(4), 1125–1146 (2018)Google Scholar - 26.He, X., et al.: Practical lessons from predicting clicks on ads at Facebook. In: Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, pp. 1–9 (2014)Google Scholar
- 27.Hoffman, M., Kahn, L.B., Li, D.: Discretion in hiring. Q. J. Econ.
**133**(2), 765–800 (2018)CrossRefGoogle Scholar - 28.Isaac, W.S.: Hope, hype, and fear: the promise and potential pitfalls of artificial intelligence in criminal justice. Ohio State J. Crim. Law
**15**(2), 543–558 (2017)MathSciNetGoogle Scholar - 29.Kleinberg, J., Mullainathan, S., Raghavan, M.: Inherent trade-offs in the fair determination of risk scores. In: 8th Innovations in Theoretical Computer Science Conference, pp. 43:1–43:23 (2016)Google Scholar
- 30.Koepke, J.L., Robinson, D.G.: Danger ahead: risk assessment and the future of bail reform. Wash. Law Rev.
**93**, 1725–1807 (2017)Google Scholar - 31.Kourou, K., Exarchos, T.P., Exarchos, K.P., Karamouzis, M.V., Fotiadis, D.I.: Machine learning applications in cancer prognosis and prediction. Comput. Struct. Biotechnol. J.
**13**, 8–17 (2015)CrossRefGoogle Scholar - 32.Kusner, M.J., Loftus, J.R., Russell, C., Silva, R.: Counterfactual fairness. In: Advances in Neural Information Processing Systems 30, pp. 4069–4079 (2017)Google Scholar
- 33.Lum, K.: Limitations of mitigating judicial bias with machine learning. Nat. Hum. Behav.
**1**(7), 1 (2017)CrossRefGoogle Scholar - 34.Lum, K., Isaac, W.: To predict and serve? Significance
**13**(5), 14–19 (2016)CrossRefGoogle Scholar - 35.Malekipirbazari, M., Aksakalli, V.: Risk assessment in social lending via random forests. Expert Syst. Appl.
**42**(10), 4621–4631 (2015)CrossRefGoogle Scholar - 36.Mayson, S.G.: Bias in, bias out. Yale Law Sch. J.
**128**(2019)Google Scholar - 37.Mitchell, S., Potash, E., Barocas, S.: Prediction-based decisions and fairness: a catalogue of choices, assumptions, and definitions (2018)Google Scholar
- 38.Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
- 39.Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. Wiley, Hoboken (2016)zbMATHGoogle Scholar
- 40.Perlich, C., Dalessandro, B., Raeder, T., Stitelman, O., Provost, F.: Machine learning for targeted display advertising: transfer learning in action. Mach. Learn.
**95**(1), 103–127 (2014)MathSciNetCrossRefGoogle Scholar - 41.Peters, J., Janzing, D., Schölkopf, B.: Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press, Cambridge (2017)zbMATHGoogle Scholar
- 42.Rosenberg, M., Levinson, R.: Trump’s catch-and-detain policy snares many who call the U.S. home, June 2018. https://www.reuters.com/investigates/special-report/usa-immigration-court
- 43.Selbst, A.D.: Disparate impact in big data policing. Georgia Law Rev.
**52**, 109–195 (2017)Google Scholar - 44.Spirtes, P., et al.: Causation, Prediction, and Search. MIT Press, Cambridge (2000)zbMATHGoogle Scholar
- 45.Stevenson, M.T.: Assessing risk assessment in action. Minnesota Law Rev.
**103**, 303 (2017)Google Scholar - 46.Vaithianathan, R., Maloney, T., Putnam-Hornstein, E., Jiang, N.: Children in the public benefit system at risk of maltreatment: identification via predictive modeling. Am. J. Prev. Med.
**45**(3), 354–359 (2013)CrossRefGoogle Scholar - 47.Zhang, J., Bareinboim, E.: Fairness in decision-making - the causal explanation formula. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)Google Scholar