Skip to main content

Why Unbiased Computational Processes Can Lead to Discriminative Decision Procedures

  • Chapter

Part of the Studies in Applied Philosophy, Epistemology and Rational Ethics book series (SAPERE,volume 3)

Abstract

Nowadays, more and more decision procedures are supported or even guided by automated processes. An important technique in this automation is data mining. In this chapter we study how such automatically generated decision support models may exhibit discriminatory behavior towards certain groups based upon, e.g., gender or ethnicity. Surprisingly, such behavior may even be observed when sensitive information is removed or suppressed and the whole procedure is guided by neutral arguments such as predictive accuracy only. The reason for this phenomenon is that most data mining methods are based upon assumptions that are not always satisfied in reality, namely, that the data is correct and represents the population well. In this chapter we discuss the implicit modeling assumptions made by most data mining algorithms and show situations in which they are not satisfied. Then we outline three realistic scenarios in which an unbiased process can lead to discriminatory models. The effects of the implicit assumptions not being fulfilled are illustrated by examples. The chapter concludes with an outline of the main challenges and problems to be solved.

Keywords

  • Training Data
  • Data Mining
  • Postal Code
  • Decision Procedure
  • Sensitive Attribute

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-30487-3_3
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-30487-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   199.99
Price excludes VAT (USA)
Hardcover Book
USD   199.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Blank, R., Dabady, M., Citro, C.: Measuring Racial Discrimination. Natl Academy Press (2004)

    Google Scholar 

  • Jonah, B.A.: Accident risk and risk-taking behavior among young drivers. Accident Analysis & Prevention 18(4), 255–271 (1986)

    CrossRef  Google Scholar 

  • Calders, T., Verwer, S.: Three Naive Bayes Approaches for Discrimination-Free Classification. Data Mining and Knowledge Discovery 21(2), 277–292 (2010)

    MathSciNet  CrossRef  Google Scholar 

  • Distance Learning Center. Internet Based Benefit and Compensation Administration: Discrimination in Pay, ch. 26 (2009), http://www.eridlc.com/index.cfm?fuseaction=textbook.chpt26 (accessed: November 2011)

  • Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons (2001)

    Google Scholar 

  • Fang, H., Moro, A.: Theories of Statistical Discrimination and Affirmative Action: A Survey. In: Benhabib, J., Bisin, A., Jackson, M. (eds.) Handbook of Social Economics, pp. 133–200 (2010)

    Google Scholar 

  • Kamiran, F., Calders, T.: Classification with no discrimination by preferential sampling. In: Proceedings of the 19th Annual Machine Learning Conference of Belgium and the Netherlands (BENELEARN 2010), pp. 1–6 (2010)

    Google Scholar 

  • Kamiran, F., Calders, T.: Classifying without Discrimination. In: IEEE International Conference on Computer, Control and Communication (IEEE-IC4), pp. 1–6 (2009)

    Google Scholar 

  • Kamiran, F., Calders, T., Pechenizkiy, M.: Discrimination Aware Decision Tree Learning. In: Proceedings of IEEE ICDM International Conference on Data Mining (ICDM 2010), pp. 869–874 (2010)

    Google Scholar 

  • Kelly, M.G., Hand, D.J., Adams, N.M.: The Impact of Changing Populations on Classifier Performance. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 1999), pp. 367–371 (1999)

    Google Scholar 

  • Rice, W.: Race, Gender, “Redlining”, and the Discriminatory Access to Loans, Credit, and Insurance: An Historical and Empirical Analysis of Consumers Who Sued Lenders and Insurers in Federal and State Courts. San Diego Law Review 33, 637–646 (1996)

    Google Scholar 

  • Turner, A., Skidmore, F.: Introduction, Summary, and Recommendations. In: Turner, A., Skidmore, F. (eds.) Mortgage Lending Discrimination: A Review of Existing Evidence (Urban Institute Monograph Series on Race and Discrimination), pp. 1–22. Urban Institute Press, Washington, DC (1999)

    Google Scholar 

  • Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23(1), 69–101 (1996)

    Google Scholar 

  • Zadrozny, B.: Learning and Evaluating Classifiers under Sample Selection Bias. In: Proceedings of the 21st International Conference on Machine Learning (ICML 2004), pp. 903–910 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Toon Calders .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Calders, T., Žliobaitė, I. (2013). Why Unbiased Computational Processes Can Lead to Discriminative Decision Procedures. In: Custers, B., Calders, T., Schermer, B., Zarsky, T. (eds) Discrimination and Privacy in the Information Society. Studies in Applied Philosophy, Epistemology and Rational Ethics, vol 3. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30487-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-30487-3_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-30486-6

  • Online ISBN: 978-3-642-30487-3

  • eBook Packages: EngineeringEngineering (R0)