Skip to main content
SpringerLink
Log in

From privacy to anti-discrimination in times of machine learning

  • Original Paper
  • Published:
Ethics and Information Technology Aims and scope Submit manuscript

Abstract

Due to the technology of machine learning, new breakthroughs are currently being achieved with constant regularity. By using machine learning techniques, computer applications can be developed and used to solve tasks that have hitherto been assumed not to be solvable by computers. If these achievements consider applications that collect and process personal data, this is typically perceived as a threat to information privacy. This paper aims to discuss applications from both fields of personality and image analysis. These applications are often criticized by reference to the protection of privacy. This paper critically questions this approach. Instead of solely using the concept of privacy to address the risks of machine learning, it is increasingly necessary to consider and implement ethical anti-discrimination concepts, too. In many ways, informational privacy requires individual information control. However, not least because of machine learning technologies, information control has become obsolete. Hence, societies need stronger anti-discrimination tenets to counteract the risks of machine learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Altman, I. (1977). Privacy regulation: Culturally universal or culturally specific? Journal of Social Issues,33(3), 66–84.

    Google Scholar 

  • Barocas, S., & Selbst, A. D. (2016). Big datas disparate impact. California Law Review,104, 671–732.

    Google Scholar 

  • Beauchamp, T. L. (2011). Informed consent: Its history, meaning, and present challenges. Cambridge Quarterly of Healthcare Ethics: CQ: The International Journal of Healthcare Ethics Committees,20(4), 515–523.

    Google Scholar 

  • Behrens, J. T. (1997). Principles and procedures of exploratory data analysis. Psychological Methods,2(2), 131–160.

    Google Scholar 

  • Belliger, A., & Krieger, D. J. (2018). Network public governance: On privacy and the informational self. Bielefeld: Transcript.

    Google Scholar 

  • Biczók, G., & Chia, P. H. (2013). Interdependent privacy: Let me share your data. Berlin: Springer.

    Google Scholar 

  • Binns, R. (2018). Fairness in machine learning: Lessons from political philosophy. In A. F. Sorelle & C. Wilson (Eds.), Conference on fairness, accountability, and transparency, New York (81st ed., pp. 1–11). PMLR.

  • Blum, A., Ligett, K., & Roth, A. (2013). A learning theory approach to noninteractive database privacy. Journal of the ACM,60(2), 1–25.

    MathSciNet  MATH  Google Scholar 

  • Bordes, A., Weston, J., & Chopra, S. (2014). Question answering with subgraph embeddings. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), October 25–29, 2014, Doha, Qatar (pp. 1–10). ACM.

  • Bowie, N. E. (2013). Privacy and the Internet. In H. LaFollette (Ed.), The international encyclopedia of ethics (pp. 4110–4114). Hoboken, NJ: Wiley-Blackwell.

    Google Scholar 

  • Boyd, D. (2008). Facebook’s Privacy Trainwreck: Exposure, invasion, and social convergence. Convergence: The International Journal of Research in New Media Technologies,14(1), 13–20.

    Google Scholar 

  • Boyd, D. (2011). Social network sites as networked publics: Affordances, dynamics and implications. In Z. Papacharissi (Ed.), A networked self: Identity, community, and culture on social network sites (pp. 39–58). New York: Routledge.

    Google Scholar 

  • Boyd, D. (2014). It’s complicated: The social lives of networked teens. New Haven, CT: Yale University Press.

    Google Scholar 

  • Brennan, T., Dieterich, W., & Ehret, B. (2008). Evaluating the predictive validity of the COMPAS risk and needs assessment system. Criminal Justice and Behavior,36, 21–40. https://doi.org/10.1177/0093854808326545.

    Article  Google Scholar 

  • Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., et al. (2018). The malicious use of artificial intelligence: Forecasting, prevention, and mitigation (pp. 1–101). https://doi.org/10.17863/CAM.22520.

  • Calders, T., & Verwer, S. (2010). Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery,21(2), 277–292.

    MathSciNet  Google Scholar 

  • Cavoukian, A. (2011). Privacy by design: The 7 foundational principles: implementation and mapping of fair information practices. https://iapp.org/media/pdf/resource_center/Privacy%20by%20Design%20-%207%20Foundational%20Principles.pdf. Accessed 21 June 2018.

  • Cavoukian, A., Taylor, S., & Abrams, M. E. (2010). Privacy by Design: Essential for organizational accountability and strong business practices. Identity in the Information Society,3(2), 405–413.

    Google Scholar 

  • Ciodaro, T., Deva, D., de Seixas, J. M., & Damazio, D. (2012). Online particle detection with Neural Networks based on topological calorimetry information. Journal of Physics: Conference Series,368, 12030. https://doi.org/10.1088/1742-6596/368/1/012030.

    Article  Google Scholar 

  • Citron, D. K., & Pasquale, F. (2014). The scored society: Due process for automated predictions. Washington Law Review,89, 1–33.

    Google Scholar 

  • Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research,12, 2493–2537.

    MATH  Google Scholar 

  • Cully, A., Clune, J., Tarapore, D., & Mouret, J.-B. (2015). Robots that can adapt like animals. Nature,521(7553), 503–507.

    Google Scholar 

  • Dhont, K., Hodson, G., Costello, K., & MacInnis, C. C. (2014). Social dominance orientation connects prejudicial human–human and human–animal relations. Personality and Individual Differences,61–62, 105–108.

    Google Scholar 

  • Diakopoulos, N., Friedler, S. A., Arenas, M., Barocas, S., Hay, M., Howe, B., et al. (2017). Principles for accountable algorithms and a social impact statement for algorithms. In Fairness, accountability, and transparency. https://www.fatml.org/resources/principles-for-accountable-algorithms. Accessed 31 July 2019.

  • Duchi, J. C., Jordan, M. I., & Wainwright, M. J. (2013). Privacy aware learning (pp. 1–60). Berkeley, CA: University of California.

    MATH  Google Scholar 

  • Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2011). Fairness through awareness (pp. 1–24). arxiv.org:1104.3913.

  • Eichstaedt, J. C., Smith, R. J., Merchant, R. M., Ungar, L. H., Crutchley, P., Preoţiuc-Pietro, D., et al. (2018). Facebook language predicts depression in medical records. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.1802331115.

    Article  Google Scholar 

  • Ekstrand, M. D., Joshaghani, R., & Mehrpouyan, H. (2018). Privacy for all: Ensuring fair and equitable privacy protections. In A. F. Sorelle & C. Wilson (Eds.), Conference on fairness, accountability, and transparency, New York (81st ed., pp. 1–13). PMLR.

  • Elgesem, D. (1996). Privacy, respect for persons, and risk. In C. Ess (Ed.), Philosophical perspectives on computer-mediated communication (pp. 45–66). New York: State University of New York Press.

    Google Scholar 

  • Floridi, L. (2005). The ontological interpretation of informational privacy. Ethics and Information Technology,7(4), 185–200.

    Google Scholar 

  • Floridi, L. (2006). Four challenges for a theory of informational privacy. Ethics and Information Technology,8(3), 109–119.

    Google Scholar 

  • Fraser, K. C., Meltzer, J. A., & Rudzicz, F. (2016). Linguistic features identify Alzheimer’s disease in narrative speech. Journal of Alzheimer’s Disease,49, 407–422.

    Google Scholar 

  • Gebru, T., Krause, J., Wang, Y., Chen, D., Deng, J., Aiden, E. L., et al. (2017). Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proceedings of the National Academy of Sciences of the United States of America,114(50), 13108–13113.

    Google Scholar 

  • Ghazinour, K., Matwin, S., & Sokolova, M. (2013). YourPrivacyProtector: A recommender system for privacy settings in social networks. International Journal of Security, Privacy and Trust Management,2(2), 11–25.

    Google Scholar 

  • Gutwirth, S., Leenes, R., & de Hert, P. (Eds.). (2015). Reforming European data protection law. Dordrecht: Springer.

    Google Scholar 

  • Hajian, S., & Domingo-Ferrer, J. (2013). Direct and indirect discrimination prevention methods. In B. Custers, T. Calders, B. Schermer, & T. Zarsky (Eds.), Discrimination and privacy in the information society: Data mining and profiling in large databases (pp. 241–256). Berlin: Springer.

    Google Scholar 

  • Haque, A., Guo, M., Miner, A. S., & Fei-Fei, L. (2018). Measuring depression symptom severity from spoken language and 3D facial expressions (pp. 1–7). https://arxiv.org/abs/1811.08592.

  • Hirsh, J. B., Kang, S. K., & Bodenhausen, G. V. (2012). Personalized persuasion: Tailoring persuasive appeals to recipients’ personality traits. Psychological Science,23(6), 578–581.

    Google Scholar 

  • Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification (pp. 1–12). https://arxiv.org/abs/1801.06146v5.

  • Hutchinson, B., & Mitchell, M. (2018). 50 Years of test (un)fairness: Lessons for machine learning (pp. 1–11). https://arxiv.org/abs/1811.10104v2.

  • Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., & Ermon, S. (2016). Combining satellite imagery and machine learning to predict poverty. Science,353(6301), 790–794.

    Google Scholar 

  • Jean, S., Cho, K., Memisevic, R., & Bengio, Y. (2015). On using very large target vocabulary for neural machine translation. In Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th international joint conference on natural language processing, Beijing, China, July 26–31, 2015 (pp.1–10). Association for Computational Linguistics.

  • Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science,349(6245), 255–260.

    MathSciNet  MATH  Google Scholar 

  • Kamiran, F., & Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems,33(1), 1–33.

    Google Scholar 

  • Kamiran, F., Calders, T., & Pechenizkiy, M. (2013). Techniques for discrimination-free predictive models. In B. Custers, T. Calders, B. Schermer, & T. Zarsky (Eds.), Discrimination and privacy in the information society: Data mining and profiling in large databases (pp. 223–239). Berlin: Springer.

    Google Scholar 

  • Khandani, A. E., Kim, A. J., & Lo, A. W. (2010). Consumer credit-risk models via machine-learning algorithms. Journal of Banking and Finance,34, 2767–2787. https://doi.org/10.1016/j.jbankfin.2010.06.001.

    Article  Google Scholar 

  • King, T. C., Aggarwal, N., Taddeo, M., & Floridi, L. (2018). Artificial intelligence crime: An interdisciplinary analysis of foreseeable threats and solutions. SSRN Electronic Journal, 1–36. https://dx.doi.org/10.2139/ssrn.3183238.

  • Kleinberg, J. M., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores (pp. 1–23). https://arxiv.org/abs/1609.05807v2.

  • Knight, W., & Hao, K. (2019). Never mind killer robots—Here are six real AI dangers to watch out for in 2019. https://www.technologyreview.com/s/612689/never-mind-killer-robotshere-are-six-real-ai-dangers-to-watch-out-for-in-2019/. Accessed 25 January 2019.

  • Kosinski, M., Bachrach, Y., Kohli, P., Stillwell, D., & Graepel, T. (2014). Manifestations of user personality in website choice and behaviour on online social networks. Machine Learning,95(3), 357–380.

    MathSciNet  Google Scholar 

  • Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences of the United States of America,110(15), 5802–5805.

    Google Scholar 

  • Kosinski, M., & Wang, Y. (2017). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Journal of Personality and Social Psychology,114, 1–47.

    Google Scholar 

  • Koskela, H. (2000). ‘The gaze without eyes’: Video-surveillance and the changing nature of urban space. Progress in Human Geography,24(2), 243–265.

    Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). ImageNet classification with deep convolutional neural networks. Proceedings of Advances in Neural Information Processing Systems,25, 1090–1098.

    Google Scholar 

  • Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017). Counterfactual fairness (pp. 1–21). https://arxiv.org/abs/1703.06856v3.

  • Lambiotte, R., & Kosinski, M. (2014). Tracking the digital footprints of personality. Proceedings of the IEEE,102(12), 1934–1939.

    Google Scholar 

  • Leuner, J. (2019). A replication study: Machine learning models are capable of predicting sexual orientation from facial images (pp. 1–69). https://arxiv.org/abs/1902.10739v1.

  • Leung, M. K. K., Xiong, H. Y., Lee, L. J., & Frey, B. J. (2014). Deep learning of the tissue-regulated splicing code. Bioinformatics,30(12), i121–i129.

    Google Scholar 

  • Li, X., Hong, X., Moilanen, A., Huang, X., Pfister, T., Zhao, G., et al. (2018). Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Transactions on Affective Computing,9(4), 563–577.

    Google Scholar 

  • Ma, J., Sheridan, R. P., Liaw, A., Dahl, G. E., & Svetnik, V. (2015). Deep neural nets as a method for quantitative structure–activity relationships. Journal of Chemical Information and Modeling,55(2), 263–274.

    Google Scholar 

  • Mackenzie, A. (2015). The production of prediction: What does machine learning want? European Journal of Cultural Studies,18(4–5), 429–445.

    Google Scholar 

  • Mahoney, J. F., & Mohen, J. M. (2007). Method and system for loan origination and underwriting. US 09/475,153.

  • Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. (2017). Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences of the United States of America,114, 1–6.

    Google Scholar 

  • McCurrie, M., Beletti, F., Parzianello, L., Westendorp, A., Anthony, S., & Scheirer, W. (2017). Predicting first impressions with deep learning (pp. 1–8). https://arxiv.org/abs/1610.08119v2.

  • McPherson, R., Shokri, R., & Shmatikov, V. (2016). Defeating image obfuscation with deep learning (pp. 1–12). https://arxiv.org/abs/1609.00408v2.

  • Mislove, A., Viswanath, B., Gummadi, K. P., & Druschel, P. (2010). You are who you know: Inferrring user profiles in online social networks. In B. D. Davison (Ed.), Proceedings of the third ACM international conference on Web search and data mining. New York: ACM.

  • Moor, J. H. (1997). Towards a theory of privacy in the information age. Computers and Society,27(3), 27–32.

    Google Scholar 

  • Moore, A. (2008). Defining privacy. Journal of Social Philosophy,39(3), 411–428.

    Google Scholar 

  • Nissenbaum, H. (2010). Privacy in context: Technology, policy, and the integrity of social life. Palo Alto, CA: Stanford University Press.

    Google Scholar 

  • Pedreschi, D., Ruggieri, S., & Turini, F. (2009). Measuring discrimination in socially-sensitive decision records. In C. Apte, H. Park, K. Wang, & M. J. Zaki (Eds.), Proceedings of the 2009 SIAM international conference on data mining (pp. 581–592). Philadelphia: Society for Industrial and Applied Mathematics.

  • Pedreshi, D., Ruggieri, S., & Turini, F. (2008). Discrimination-aware data mining. In Y. Li, B. Liu, & S. Sarawagi (Eds.), 14th ACM SIGKDD international conference, Las Vegas (pp. 560–568). New York: ACM Press.

  • Pu, Y., & Grossklags, J. (2016). Towards a model on the factors influencing social app users’ valuation of interdependent privacy. Proceedings on Privacy Enhancing Technologies (2), 61–81. https://doi.org/10.1515/popets-2016-0005.

    Google Scholar 

  • Regan, P. M. (1995). Legislating privacy: Technology, social values, and public policy. Chapel Hill, NC: University of North Carolina Press.

    Google Scholar 

  • Rössler, B. (2001). Der Wert des Privaten. Frankfurt am Main: Suhrkamp.

    Google Scholar 

  • Sajjadi, M. S. M., Schölkopf, B., & Hirsch, M. (2017). EnhanceNet: Single image super-resolution through automated texture synthesis (pp. 1–19). https://arxiv.org/abs/1612.07919v2.

  • Sarigol, E., Garcia, D., & Schweitzer, F. (2014). Online privacy as a collective phenomenon (pp. 1–11). https://arxiv.org/abs/1409.6197v1.

  • Sarwate, A. D., & Chaudhuri, K. (2013). Signal processing and machine learning with differential privacy: Algorithms and challenges for continuous data. IEEE Signal Processing Magazine,30(5), 86–94.

    Google Scholar 

  • Schneier, B. (2015). Das Ende der Geheimnisse. Technology Review (12), 76–77.

  • Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2018). Fairness and abstraction in sociotechnical systems. In ACT conference on fairness, accountability, and transparency (FAT) (Vol. 1(1), pp. 1–17).

  • Sermanet, P., Kavukcuoglu, K., Chintala, S., & LeCun, Y. (2012). Pedestrian detection with unsupervised multi-stage feature learning (pp. 1–12). https://arxiv.org/abs/1212.0142v2.

  • Shillingford, B., Assael, Y., Hoffman, M. W., Paine, T., Hughes, C., Prabhu, U., et al. (2018). Large-scale visual speech recognition (pp. 1–21). https://arxiv.org/abs/1807.05162v3.

  • Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., et al. (2017). Mastering the game of Go without human knowledge. Nature,550(7676), 354–359.

    Google Scholar 

  • Surden, H. (2014). Machine learning and law. Washington Law Review,89(1), 87–115.

    Google Scholar 

  • Susser, D. (2016). Information privacy and social self-authorship. Techné: Research in Philosophy and Technology. https://doi.org/10.2139/ssrn.2706669.

    Article  Google Scholar 

  • Taigman, Y., Yang, M., Ranzato, M.’ A., & Wolf, L. (2014). DeepFace: Closing the gap to human-level performance in face verification. In 2014 IEEE conference on computer vision and pattern recognition (CVPR), Columbus, Ohio (pp. 1701–1708). IEEE.

  • Tavani, H. T. (2007). Philosophical theories of privacy: Implications for an adequate online privacy policy. Metaphilosophy,38(1), 1–22.

    Google Scholar 

  • Tavani, H. T. (2008). Informational privacy: Concepts, theories, and controversies. In K. E. Himma & H. T. Tavani (Eds.), The handbook of information and computer ethics (pp. 131–164). Hoboken, NJ: Wiley.

    Google Scholar 

  • Tavani, H. T., & Moor, J. H. (2001). Privacy protection, control of information, and privacy-enhancing technologies. ACM SIGCAS Computers and Society,31(1), 6–11.

    Google Scholar 

  • Tutt, A. (2017). An FDA for algorithms. Administrative Law Review,83, 83–123.

    Google Scholar 

  • Van den Hoven, J. (1997). Privacy and the varieties of moral wrong-doing in an information age. Computers and Society,27, 33–37.

    Google Scholar 

  • Van den Hoven, J. (2001). Privacy and the varieties of informational wrongdoing. In Readings in Cyberethics (pp. 430–442). London: Taylor and Francis.

  • Veale, M., & Binns, R. (2017). Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data and Society,4(2), 1–17.

    Google Scholar 

  • Vedder, A., & Naudts, L. (2017). Accountability for the use of algorithms in a big data environment. International Review of Law, Computers and Technology,31(2), 206–224.

    Google Scholar 

  • Westin, A. F. (1967). Privacy and freedom. New York: Atheneum.

    Google Scholar 

  • Wheeler, G. (2017). Machine epistemology and big data. In L. C. McIntyre & A. Rosenberg (Eds.), The Routledge companion to philosophy of social science (pp. 1–11). London: Routledge Taylor and Francis Group.

    Google Scholar 

  • Wu, X., & Zhang, X. (2016). Responses to critiques on machine learning of criminality perceptions (Addendum of arXiv:1611.04135) (pp. 1–14). https://arxiv.org/abs/1611.04135v3.

  • Xiong, H. Y., Alipanahi, B., Lee, L. J., Bretschneider, H., Merico, D., Yuen, R. K. C., et al. (2015). RNA splicing. The human splicing code reveals new insights into the genetic determinants of disease. Science,347(6218), 1–20.

    Google Scholar 

  • Young, I. M. (1990). Justice and the politics of difference. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Youyou, W., Kosinski, M., & Stillwell, D. (2015). Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences of the United States of America,112(4), 1036–1040.

    Google Scholar 

  • Zafar, M. B., Valera, I., Rodriguez, M. G., & Gummadi, K. P. (2017). Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment (pp. 1–10). https://arxiv.org/abs/1610.08452v2.

Download references

Acknowledgements

This research was supported by the Cluster of Excellence “Machine Learning – New Perspectives for Science” funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy: Reference Number EXC 2064/1: Project ID 390727645.

Author information

Authors and Affiliations

  1. International Centre for Ethics in the Sciences and Humanities, University of Tuebingen, Wilhelmstraße 19, 72074, Tübingen, Germany

    Thilo Hagendorff

Authors
  1. Thilo Hagendorff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thilo Hagendorff.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hagendorff, T. From privacy to anti-discrimination in times of machine learning. Ethics Inf Technol 21, 331–343 (2019). https://doi.org/10.1007/s10676-019-09510-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10676-019-09510-5

Keywords

Search

Navigation