Skip to main content
SpringerLink
Log in

Fair Outlier Detection

  • Conference paper
  • First Online:
Web Information Systems Engineering – WISE 2020 (WISE 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12343))

Included in the following conference series:

Abstract

An outlier detection method may be considered fair over specified sensitive attributes if the results of outlier detection are not skewed towards particular groups defined on such sensitive attributes. In this paper, we consider, for the first time to our best knowledge, the task of fair outlier detection. Our focus is on the task of fair outlier detection over multiple multi-valued sensitive attributes (e.g., gender, race, religion, nationality, marital status etc.), one that has broad applications across web data scenarios. We propose a fair outlier detection method, FairLOF, that is inspired by the popular LOF formulation for neighborhood-based outlier detection. We outline ways in which unfairness could be induced within LOF and develop three heuristic principles to enhance fairness, which form the basis of the FairLOF method. Being a novel task, we develop an evaluation framework for fair outlier detection, and use that to benchmark FairLOF on quality and fairness of results. Through an extensive empirical evaluation over real-world datasets, we illustrate that FairLOF is able to achieve significant improvements in fairness at sometimes marginal degradations on result quality as measured against the fairness-agnostic LOF method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Change history

  • 16 December 2020

    The original version of this chapter was revised. The first author’s name was corrected to Deepak P.

Notes

  1. 1.

    https://www.nyclu.org/en/stop-and-frisk-data.

  2. 2.

    https://www.cnet.com/features/is-facebook-censoring-conservatives-or-is-moderating-just-too-hard/.

  3. 3.

    \(|N_k(X)|\) could be greater than k in case there is a tie for the \(k^{th}\) place.

  4. 4.

    https://www.statista.com/statistics/382525/share-of-police-officers-in-england-and-wales-gender-rank/.

  5. 5.

    https://en.wikipedia.org/wiki/Reservation_in_India.

  6. 6.

    https://en.wikipedia.org/wiki/AOL_search_data_leak.

References

  1. Abraham, S.S., Deepak, P., Sundaram, S.S.: Fairness in clustering with multiple sensitive attributes. In: EDBT, pp. 287–298 (2020)

    Google Scholar 

  2. Asudeh, A., Jagadish, H., Stoyanovich, J., Das, G.: Designing fair ranking schemes. In: SIGMOD (2019)

    Google Scholar 

  3. Babaei, K., Chen, Z., Maul, T.: Detecting point outliers using prune-based outlier factor (PLOF). arXiv preprint arXiv:1911.01654 (2019)

  4. Barocas, S., Selbst, A.D.: Big data’s disparate impact. Calif. Law. Rev. 104, 671 (2016)

    Google Scholar 

  5. Bei, X., Liu, S., Poon, C.K., Wang, H.: Candidate selections with proportional fairness constraints. In: AAMAS (2020)

    Google Scholar 

  6. Bera, S.K., Chakrabarty, D., Flores, N., Negahbani, M.: Fair algorithms for clustering. In: NeurIPS, pp. 4955–4966 (2019)

    Google Scholar 

  7. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: SIGMOD, pp. 93–104 (2000)

    Google Scholar 

  8. Chandola, V., Banerjee, A., Kumar, V.: Outlier detection: a survey. ACM Comput. Surv. 14, 15 (2007)

    Google Scholar 

  9. Chawla, S., Sun, P.: SLOM: a new measure for local spatial outliers. Knowl. Inf. Syst. 9(4), 412–429 (2006). https://doi.org/10.1007/s10115-005-0200-2

    Article  Google Scholar 

  10. Chen, J., Sathe, S., Aggarwal, C., Turaga, D.: Outlier detection with autoencoder ensembles. In: SDM (2017)

    Google Scholar 

  11. Chierichetti, F., Kumar, R., Lattanzi, S., Vassilvitskii, S.: Fair clustering through fairlets. In: NIPS (2017)

    Google Scholar 

  12. Chouldechova, A., Roth, A.: A snapshot of the frontiers of fairness in machine learning. Commun. ACM 63(5), 82–89 (2020)

    Article  Google Scholar 

  13. Davidson, I., Ravi, S.: A framework for determining the fairness of outlier detection. In: ECAI (2020)

    Google Scholar 

  14. Domingues, R., Filippone, M., Michiardi, P., Zouaoui, J.: A comparative evaluation of outlier detection algorithms: experiments and analyses. Pattern Recogn. 74, 406–421 (2018)

    Article  Google Scholar 

  15. Fan, W., Bouguila, N., Ziou, D.: Unsupervised anomaly intrusion detection via localized Bayesian feature selection. In: ICDM (2011)

    Google Scholar 

  16. Hawkins, D.M.: Identification of Outliers. MSAP, vol. 11. Springer, Dordrecht (1980). https://doi.org/10.1007/978-94-015-3994-4

    Book  MATH  Google Scholar 

  17. Huang, L., Vishnoi, N.K.: Stable and fair classification. arXiv:1902.07823 (2019)

  18. Jabez, J., Muthukumar, B.: Intrusion detection system (IDS): anomaly detection using outlier detection approach. Procedia Comput. Sci. 48, 338–346 (2015)

    Article  Google Scholar 

  19. Knight, C.: Luck Egalitarianism: Equality, Responsibility, and Justice. EUP, Edinburgh (2009)

    Google Scholar 

  20. Kriegel, H.P., Kröger, P., Schubert, E., Zimek, A.: LoOP: local outlier probabilities. In: CIKM (2009)

    Google Scholar 

  21. Kumar, V., Kumar, D., Singh, R.: Outlier mining in medical databases: an application of data mining in health care management to detect abnormal values presented in medical databases. IJCSNS Int. J. Comput. Sci. Netw. Secur. 8, 272–277 (2008)

    Google Scholar 

  22. Olfat, M., Aswani, A.: Convex formulations for fair principal component analysis. In: AAAI, vol. 33, pp. 663–670 (2019)

    Google Scholar 

  23. Deepak, P.: Whither fair clustering? In: AI for Social Good Workshop (2020)

    Google Scholar 

  24. Patro, G.K., et al.: Incremental fairness in two-sided market platforms: on updating recommendations fairly. In: AAAI (2020)

    Google Scholar 

  25. Pawar, A.D., Kalavadekar, P.N., Tambe, S.N.: A survey on outlier detection techniques for credit card fraud detection. IOSR J. Comput. Eng. 16(2), 44–48 (2014)

    Article  Google Scholar 

  26. Pearson, K.: Vii. Note on regression and inheritance in the case of two parents. Proc. R. Soc. London 58(347–352), 240–242 (1895)

    Google Scholar 

  27. Rawls, J.: A Theory of Justice. Harvard University Press, Cambridge (1971)

    Google Scholar 

  28. Schubert, E., Zimek, A., Kriegel, H.-P.: Local outlier detection reconsidered: a generalized view on locality with applications to spatial, video, and network outlier detection. Data Min. Knowl. Disc. 28(1), 190–237 (2012). https://doi.org/10.1007/s10618-012-0300-z

    Article  MathSciNet  MATH  Google Scholar 

  29. Wang, B., Davidson, I.: Towards fair deep clustering with multi-state protected variables. arXiv preprint arXiv:1901.10053 (2019)

  30. Yu, D., Sheikholeslami, G., Zhang, A.: FindOut: finding outliers in very large datasets. Knowl. Inf. Syst. 4(4), 387–412 (2002). https://doi.org/10.1007/s101150200013

    Article  Google Scholar 

  31. Zafar, M.B., Valera, I., Rodriguez, M.G., Gummadi, K.P.: Fairness constraints: mechanisms for fair classification. arXiv preprint arXiv:1507.05259 (2015)

  32. Zehlike, M., Bonchi, F., Castillo, C., Hajian, S., Megahed, M., Baeza-Yates, R.: FA*IR: a fair top-k ranking algorithm. In: CIKM, pp. 1569–1578 (2017)

    Google Scholar 

  33. Zhang, K., Hutter, M., Jin, H.: A new local distance-based outlier detection approach for scattered real-world data. In: PAKDD (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Queen’s University Belfast, Belfast, UK

    Deepak P.

  2. Indian Institute of Technology Madras, Chennai, India

    Deepak P. & Savitha Sam Abraham

Authors
  1. Deepak P.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Savitha Sam Abraham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepak P. .

Editor information

Editors and Affiliations

  1. VU Amsterdam, Amsterdam, The Netherlands

    Zhisheng Huang

  2. VU Amsterdam, Amsterdam, The Netherlands

    Wouter Beek

  3. Victoria University, Melbourne, VIC, Australia

    Hua Wang

  4. Swinburne University of Technology, Hawthorn, VIC, Australia

    Rui Zhou

  5. Victoria University, Melbourne, VIC, Australia

    Yanchun Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

P., D., Sam Abraham, S. (2020). Fair Outlier Detection. In: Huang, Z., Beek, W., Wang, H., Zhou, R., Zhang, Y. (eds) Web Information Systems Engineering – WISE 2020. WISE 2020. Lecture Notes in Computer Science(), vol 12343. Springer, Cham. https://doi.org/10.1007/978-3-030-62008-0_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-62008-0_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-62007-3

  • Online ISBN: 978-3-030-62008-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation