From privacy to anti-discrimination in times of machine learning

Abstract

Due to the technology of machine learning, new breakthroughs are currently being achieved with constant regularity. By using machine learning techniques, computer applications can be developed and used to solve tasks that have hitherto been assumed not to be solvable by computers. If these achievements consider applications that collect and process personal data, this is typically perceived as a threat to information privacy. This paper aims to discuss applications from both fields of personality and image analysis. These applications are often criticized by reference to the protection of privacy. This paper critically questions this approach. Instead of solely using the concept of privacy to address the risks of machine learning, it is increasingly necessary to consider and implement ethical anti-discrimination concepts, too. In many ways, informational privacy requires individual information control. However, not least because of machine learning technologies, information control has become obsolete. Hence, societies need stronger anti-discrimination tenets to counteract the risks of machine learning.

This is a preview of subscription content, log in to check access.

Fig. 1

References

  1. Altman, I. (1977). Privacy regulation: Culturally universal or culturally specific? Journal of Social Issues,33(3), 66–84.

    Google Scholar 

  2. Barocas, S., & Selbst, A. D. (2016). Big datas disparate impact. California Law Review,104, 671–732.

    Google Scholar 

  3. Beauchamp, T. L. (2011). Informed consent: Its history, meaning, and present challenges. Cambridge Quarterly of Healthcare Ethics: CQ: The International Journal of Healthcare Ethics Committees,20(4), 515–523.

    Google Scholar 

  4. Behrens, J. T. (1997). Principles and procedures of exploratory data analysis. Psychological Methods,2(2), 131–160.

    Google Scholar 

  5. Belliger, A., & Krieger, D. J. (2018). Network public governance: On privacy and the informational self. Bielefeld: Transcript.

    Google Scholar 

  6. Biczók, G., & Chia, P. H. (2013). Interdependent privacy: Let me share your data. Berlin: Springer.

    Google Scholar 

  7. Binns, R. (2018). Fairness in machine learning: Lessons from political philosophy. In A. F. Sorelle & C. Wilson (Eds.), Conference on fairness, accountability, and transparency, New York (81st ed., pp. 1–11). PMLR.

  8. Blum, A., Ligett, K., & Roth, A. (2013). A learning theory approach to noninteractive database privacy. Journal of the ACM,60(2), 1–25.

    MathSciNet  MATH  Google Scholar 

  9. Bordes, A., Weston, J., & Chopra, S. (2014). Question answering with subgraph embeddings. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), October 25–29, 2014, Doha, Qatar (pp. 1–10). ACM.

  10. Bowie, N. E. (2013). Privacy and the Internet. In H. LaFollette (Ed.), The international encyclopedia of ethics (pp. 4110–4114). Hoboken, NJ: Wiley-Blackwell.

    Google Scholar 

  11. Boyd, D. (2008). Facebook’s Privacy Trainwreck: Exposure, invasion, and social convergence. Convergence: The International Journal of Research in New Media Technologies,14(1), 13–20.

    Google Scholar 

  12. Boyd, D. (2011). Social network sites as networked publics: Affordances, dynamics and implications. In Z. Papacharissi (Ed.), A networked self: Identity, community, and culture on social network sites (pp. 39–58). New York: Routledge.

    Google Scholar 

  13. Boyd, D. (2014). It’s complicated: The social lives of networked teens. New Haven, CT: Yale University Press.

    Google Scholar 

  14. Brennan, T., Dieterich, W., & Ehret, B. (2008). Evaluating the predictive validity of the COMPAS risk and needs assessment system. Criminal Justice and Behavior,36, 21–40. https://doi.org/10.1177/0093854808326545.

    Article  Google Scholar 

  15. Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., et al. (2018). The malicious use of artificial intelligence: Forecasting, prevention, and mitigation (pp. 1–101). https://doi.org/10.17863/CAM.22520.

  16. Calders, T., & Verwer, S. (2010). Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery,21(2), 277–292.

    MathSciNet  Google Scholar 

  17. Cavoukian, A. (2011). Privacy by design: The 7 foundational principles: implementation and mapping of fair information practices. https://iapp.org/media/pdf/resource_center/Privacy%20by%20Design%20-%207%20Foundational%20Principles.pdf. Accessed 21 June 2018.

  18. Cavoukian, A., Taylor, S., & Abrams, M. E. (2010). Privacy by Design: Essential for organizational accountability and strong business practices. Identity in the Information Society,3(2), 405–413.

    Google Scholar 

  19. Ciodaro, T., Deva, D., de Seixas, J. M., & Damazio, D. (2012). Online particle detection with Neural Networks based on topological calorimetry information. Journal of Physics: Conference Series,368, 12030. https://doi.org/10.1088/1742-6596/368/1/012030.

    Article  Google Scholar 

  20. Citron, D. K., & Pasquale, F. (2014). The scored society: Due process for automated predictions. Washington Law Review,89, 1–33.

    Google Scholar 

  21. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research,12, 2493–2537.

    MATH  Google Scholar 

  22. Cully, A., Clune, J., Tarapore, D., & Mouret, J.-B. (2015). Robots that can adapt like animals. Nature,521(7553), 503–507.

    Google Scholar 

  23. Dhont, K., Hodson, G., Costello, K., & MacInnis, C. C. (2014). Social dominance orientation connects prejudicial human–human and human–animal relations. Personality and Individual Differences,61–62, 105–108.

    Google Scholar 

  24. Diakopoulos, N., Friedler, S. A., Arenas, M., Barocas, S., Hay, M., Howe, B., et al. (2017). Principles for accountable algorithms and a social impact statement for algorithms. In Fairness, accountability, and transparency. https://www.fatml.org/resources/principles-for-accountable-algorithms. Accessed 31 July 2019.

  25. Duchi, J. C., Jordan, M. I., & Wainwright, M. J. (2013). Privacy aware learning (pp. 1–60). Berkeley, CA: University of California.

    Google Scholar 

  26. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2011). Fairness through awareness (pp. 1–24). arxiv.org:1104.3913.

  27. Eichstaedt, J. C., Smith, R. J., Merchant, R. M., Ungar, L. H., Crutchley, P., Preoţiuc-Pietro, D., et al. (2018). Facebook language predicts depression in medical records. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.1802331115.

    Article  Google Scholar 

  28. Ekstrand, M. D., Joshaghani, R., & Mehrpouyan, H. (2018). Privacy for all: Ensuring fair and equitable privacy protections. In A. F. Sorelle & C. Wilson (Eds.), Conference on fairness, accountability, and transparency, New York (81st ed., pp. 1–13). PMLR.

  29. Elgesem, D. (1996). Privacy, respect for persons, and risk. In C. Ess (Ed.), Philosophical perspectives on computer-mediated communication (pp. 45–66). New York: State University of New York Press.

    Google Scholar 

  30. Floridi, L. (2005). The ontological interpretation of informational privacy. Ethics and Information Technology,7(4), 185–200.

    Google Scholar 

  31. Floridi, L. (2006). Four challenges for a theory of informational privacy. Ethics and Information Technology,8(3), 109–119.

    Google Scholar 

  32. Fraser, K. C., Meltzer, J. A., & Rudzicz, F. (2016). Linguistic features identify Alzheimer’s disease in narrative speech. Journal of Alzheimer’s Disease,49, 407–422.

    Google Scholar 

  33. Gebru, T., Krause, J., Wang, Y., Chen, D., Deng, J., Aiden, E. L., et al. (2017). Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proceedings of the National Academy of Sciences of the United States of America,114(50), 13108–13113.

    Google Scholar 

  34. Ghazinour, K., Matwin, S., & Sokolova, M. (2013). YourPrivacyProtector: A recommender system for privacy settings in social networks. International Journal of Security, Privacy and Trust Management,2(2), 11–25.

    Google Scholar 

  35. Gutwirth, S., Leenes, R., & de Hert, P. (Eds.). (2015). Reforming European data protection law. Dordrecht: Springer.

    Google Scholar 

  36. Hajian, S., & Domingo-Ferrer, J. (2013). Direct and indirect discrimination prevention methods. In B. Custers, T. Calders, B. Schermer, & T. Zarsky (Eds.), Discrimination and privacy in the information society: Data mining and profiling in large databases (pp. 241–256). Berlin: Springer.

    Google Scholar 

  37. Haque, A., Guo, M., Miner, A. S., & Fei-Fei, L. (2018). Measuring depression symptom severity from spoken language and 3D facial expressions (pp. 1–7). https://arxiv.org/abs/1811.08592.

  38. Hirsh, J. B., Kang, S. K., & Bodenhausen, G. V. (2012). Personalized persuasion: Tailoring persuasive appeals to recipients’ personality traits. Psychological Science,23(6), 578–581.

    Google Scholar 

  39. Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification (pp. 1–12). https://arxiv.org/abs/1801.06146v5.

  40. Hutchinson, B., & Mitchell, M. (2018). 50 Years of test (un)fairness: Lessons for machine learning (pp. 1–11). https://arxiv.org/abs/1811.10104v2.

  41. Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., & Ermon, S. (2016). Combining satellite imagery and machine learning to predict poverty. Science,353(6301), 790–794.

    Google Scholar 

  42. Jean, S., Cho, K., Memisevic, R., & Bengio, Y. (2015). On using very large target vocabulary for neural machine translation. In Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th international joint conference on natural language processing, Beijing, China, July 26–31, 2015 (pp.1–10). Association for Computational Linguistics.

  43. Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science,349(6245), 255–260.

    MathSciNet  MATH  Google Scholar 

  44. Kamiran, F., & Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems,33(1), 1–33.

    Google Scholar 

  45. Kamiran, F., Calders, T., & Pechenizkiy, M. (2013). Techniques for discrimination-free predictive models. In B. Custers, T. Calders, B. Schermer, & T. Zarsky (Eds.), Discrimination and privacy in the information society: Data mining and profiling in large databases (pp. 223–239). Berlin: Springer.

    Google Scholar 

  46. Khandani, A. E., Kim, A. J., & Lo, A. W. (2010). Consumer credit-risk models via machine-learning algorithms. Journal of Banking and Finance,34, 2767–2787. https://doi.org/10.1016/j.jbankfin.2010.06.001.

    Article  Google Scholar 

  47. King, T. C., Aggarwal, N., Taddeo, M., & Floridi, L. (2018). Artificial intelligence crime: An interdisciplinary analysis of foreseeable threats and solutions. SSRN Electronic Journal, 1–36. https://dx.doi.org/10.2139/ssrn.3183238.

  48. Kleinberg, J. M., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores (pp. 1–23). https://arxiv.org/abs/1609.05807v2.

  49. Knight, W., & Hao, K. (2019). Never mind killer robots—Here are six real AI dangers to watch out for in 2019. https://www.technologyreview.com/s/612689/never-mind-killer-robotshere-are-six-real-ai-dangers-to-watch-out-for-in-2019/. Accessed 25 January 2019.

  50. Kosinski, M., Bachrach, Y., Kohli, P., Stillwell, D., & Graepel, T. (2014). Manifestations of user personality in website choice and behaviour on online social networks. Machine Learning,95(3), 357–380.

    MathSciNet  Google Scholar 

  51. Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences of the United States of America,110(15), 5802–5805.

    Google Scholar 

  52. Kosinski, M., & Wang, Y. (2017). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Journal of Personality and Social Psychology,114, 1–47.

    Google Scholar 

  53. Koskela, H. (2000). ‘The gaze without eyes’: Video-surveillance and the changing nature of urban space. Progress in Human Geography,24(2), 243–265.

    Google Scholar 

  54. Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). ImageNet classification with deep convolutional neural networks. Proceedings of Advances in Neural Information Processing Systems,25, 1090–1098.

    Google Scholar 

  55. Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017). Counterfactual fairness (pp. 1–21). https://arxiv.org/abs/1703.06856v3.

  56. Lambiotte, R., & Kosinski, M. (2014). Tracking the digital footprints of personality. Proceedings of the IEEE,102(12), 1934–1939.

    Google Scholar 

  57. Leuner, J. (2019). A replication study: Machine learning models are capable of predicting sexual orientation from facial images (pp. 1–69). https://arxiv.org/abs/1902.10739v1.

  58. Leung, M. K. K., Xiong, H. Y., Lee, L. J., & Frey, B. J. (2014). Deep learning of the tissue-regulated splicing code. Bioinformatics,30(12), i121–i129.

    Google Scholar 

  59. Li, X., Hong, X., Moilanen, A., Huang, X., Pfister, T., Zhao, G., et al. (2018). Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Transactions on Affective Computing,9(4), 563–577.

    Google Scholar 

  60. Ma, J., Sheridan, R. P., Liaw, A., Dahl, G. E., & Svetnik, V. (2015). Deep neural nets as a method for quantitative structure–activity relationships. Journal of Chemical Information and Modeling,55(2), 263–274.

    Google Scholar 

  61. Mackenzie, A. (2015). The production of prediction: What does machine learning want? European Journal of Cultural Studies,18(4–5), 429–445.

    Google Scholar 

  62. Mahoney, J. F., & Mohen, J. M. (2007). Method and system for loan origination and underwriting. US 09/475,153.

  63. Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. (2017). Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences of the United States of America,114, 1–6.

    Google Scholar 

  64. McCurrie, M., Beletti, F., Parzianello, L., Westendorp, A., Anthony, S., & Scheirer, W. (2017). Predicting first impressions with deep learning (pp. 1–8). https://arxiv.org/abs/1610.08119v2.

  65. McPherson, R., Shokri, R., & Shmatikov, V. (2016). Defeating image obfuscation with deep learning (pp. 1–12). https://arxiv.org/abs/1609.00408v2.

  66. Mislove, A., Viswanath, B., Gummadi, K. P., & Druschel, P. (2010). You are who you know: Inferrring user profiles in online social networks. In B. D. Davison (Ed.), Proceedings of the third ACM international conference on Web search and data mining. New York: ACM.

  67. Moor, J. H. (1997). Towards a theory of privacy in the information age. Computers and Society,27(3), 27–32.

    Google Scholar 

  68. Moore, A. (2008). Defining privacy. Journal of Social Philosophy,39(3), 411–428.

    Google Scholar 

  69. Nissenbaum, H. (2010). Privacy in context: Technology, policy, and the integrity of social life. Palo Alto, CA: Stanford University Press.

    Google Scholar 

  70. Pedreschi, D., Ruggieri, S., & Turini, F. (2009). Measuring discrimination in socially-sensitive decision records. In C. Apte, H. Park, K. Wang, & M. J. Zaki (Eds.), Proceedings of the 2009 SIAM international conference on data mining (pp. 581–592). Philadelphia: Society for Industrial and Applied Mathematics.

  71. Pedreshi, D., Ruggieri, S., & Turini, F. (2008). Discrimination-aware data mining. In Y. Li, B. Liu, & S. Sarawagi (Eds.), 14th ACM SIGKDD international conference, Las Vegas (pp. 560–568). New York: ACM Press.

  72. Pu, Y., & Grossklags, J. (2016). Towards a model on the factors influencing social app users’ valuation of interdependent privacy. Proceedings on Privacy Enhancing Technologies (2), 61–81. https://doi.org/10.1515/popets-2016-0005.

    Google Scholar 

  73. Regan, P. M. (1995). Legislating privacy: Technology, social values, and public policy. Chapel Hill, NC: University of North Carolina Press.

    Google Scholar 

  74. Rössler, B. (2001). Der Wert des Privaten. Frankfurt am Main: Suhrkamp.

    Google Scholar 

  75. Sajjadi, M. S. M., Schölkopf, B., & Hirsch, M. (2017). EnhanceNet: Single image super-resolution through automated texture synthesis (pp. 1–19). https://arxiv.org/abs/1612.07919v2.

  76. Sarigol, E., Garcia, D., & Schweitzer, F. (2014). Online privacy as a collective phenomenon (pp. 1–11). https://arxiv.org/abs/1409.6197v1.

  77. Sarwate, A. D., & Chaudhuri, K. (2013). Signal processing and machine learning with differential privacy: Algorithms and challenges for continuous data. IEEE Signal Processing Magazine,30(5), 86–94.

    Google Scholar 

  78. Schneier, B. (2015). Das Ende der Geheimnisse. Technology Review (12), 76–77.

  79. Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2018). Fairness and abstraction in sociotechnical systems. In ACT conference on fairness, accountability, and transparency (FAT) (Vol. 1(1), pp. 1–17).

  80. Sermanet, P., Kavukcuoglu, K., Chintala, S., & LeCun, Y. (2012). Pedestrian detection with unsupervised multi-stage feature learning (pp. 1–12). https://arxiv.org/abs/1212.0142v2.

  81. Shillingford, B., Assael, Y., Hoffman, M. W., Paine, T., Hughes, C., Prabhu, U., et al. (2018). Large-scale visual speech recognition (pp. 1–21). https://arxiv.org/abs/1807.05162v3.

  82. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., et al. (2017). Mastering the game of Go without human knowledge. Nature,550(7676), 354–359.

    Google Scholar 

  83. Surden, H. (2014). Machine learning and law. Washington Law Review,89(1), 87–115.

    Google Scholar 

  84. Susser, D. (2016). Information privacy and social self-authorship. Techné: Research in Philosophy and Technology. https://doi.org/10.2139/ssrn.2706669.

    Article  Google Scholar 

  85. Taigman, Y., Yang, M., Ranzato, M.’ A., & Wolf, L. (2014). DeepFace: Closing the gap to human-level performance in face verification. In 2014 IEEE conference on computer vision and pattern recognition (CVPR), Columbus, Ohio (pp. 1701–1708). IEEE.

  86. Tavani, H. T. (2007). Philosophical theories of privacy: Implications for an adequate online privacy policy. Metaphilosophy,38(1), 1–22.

    Google Scholar 

  87. Tavani, H. T. (2008). Informational privacy: Concepts, theories, and controversies. In K. E. Himma & H. T. Tavani (Eds.), The handbook of information and computer ethics (pp. 131–164). Hoboken, NJ: Wiley.

    Google Scholar 

  88. Tavani, H. T., & Moor, J. H. (2001). Privacy protection, control of information, and privacy-enhancing technologies. ACM SIGCAS Computers and Society,31(1), 6–11.

    Google Scholar 

  89. Tutt, A. (2017). An FDA for algorithms. Administrative Law Review,83, 83–123.

    Google Scholar 

  90. Van den Hoven, J. (1997). Privacy and the varieties of moral wrong-doing in an information age. Computers and Society,27, 33–37.

    Google Scholar 

  91. Van den Hoven, J. (2001). Privacy and the varieties of informational wrongdoing. In Readings in Cyberethics (pp. 430–442). London: Taylor and Francis.

  92. Veale, M., & Binns, R. (2017). Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data and Society,4(2), 1–17.

    Google Scholar 

  93. Vedder, A., & Naudts, L. (2017). Accountability for the use of algorithms in a big data environment. International Review of Law, Computers and Technology,31(2), 206–224.

    Google Scholar 

  94. Westin, A. F. (1967). Privacy and freedom. New York: Atheneum.

    Google Scholar 

  95. Wheeler, G. (2017). Machine epistemology and big data. In L. C. McIntyre & A. Rosenberg (Eds.), The Routledge companion to philosophy of social science (pp. 1–11). London: Routledge Taylor and Francis Group.

    Google Scholar 

  96. Wu, X., & Zhang, X. (2016). Responses to critiques on machine learning of criminality perceptions (Addendum of arXiv:1611.04135) (pp. 1–14). https://arxiv.org/abs/1611.04135v3.

  97. Xiong, H. Y., Alipanahi, B., Lee, L. J., Bretschneider, H., Merico, D., Yuen, R. K. C., et al. (2015). RNA splicing. The human splicing code reveals new insights into the genetic determinants of disease. Science,347(6218), 1–20.

    Google Scholar 

  98. Young, I. M. (1990). Justice and the politics of difference. Princeton, NJ: Princeton University Press.

    Google Scholar 

  99. Youyou, W., Kosinski, M., & Stillwell, D. (2015). Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences of the United States of America,112(4), 1036–1040.

    Google Scholar 

  100. Zafar, M. B., Valera, I., Rodriguez, M. G., & Gummadi, K. P. (2017). Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment (pp. 1–10). https://arxiv.org/abs/1610.08452v2.

Download references

Acknowledgements

This research was supported by the Cluster of Excellence “Machine Learning – New Perspectives for Science” funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy: Reference Number EXC 2064/1: Project ID 390727645.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Thilo Hagendorff.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hagendorff, T. From privacy to anti-discrimination in times of machine learning. Ethics Inf Technol 21, 331–343 (2019). https://doi.org/10.1007/s10676-019-09510-5

Download citation

Keywords

  • Artificial intelligence
  • Machine learning
  • Privacy
  • Discrimination
  • Fairness
  • Algorithms
  • Image analysis
  • Personality analysis