Skip to main content
SpringerLink
Log in

MindSet: A Bias-Detection Interface Using a Visual Human-in-the-Loop Workflow

  • Conference paper
  • First Online:
Artificial Intelligence. ECAI 2023 International Workshops (ECAI 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1948))

Included in the following conference series:

  • 374 Accesses

Abstract

Handling data artifacts is a critical and unsolved challenge in deep learning. Disregarding such asymmetries may lead to biased and socially unfair predictions, prohibiting applications in high-stake scenarios. In the case of visual data, its inherently unstructured nature makes automated bias detection especially difficult. Thus, a promising remedy is to rely on human feedback. Hu et al. [14] introduced a three-stage theoretical study framework to use a human-in-the-loop approach for bias detection in visual datasets and ran a small-sample study. While showing encouraging results, no implementation is available to enable researchers and practitioners to study their image datasets. In this work, we present a dataset-agnostic implementation based on a highly flexible web app interface. With this implementation, we aim to bring this theoretical framework into practice by following a user-centric approach. We also extend the framework so that the workflow can be adjusted to the researcher’s needs in terms of the granularity of detected anomalies.

S. Kalananthan, A. Kichutkin, Z. Shang and A. Strausz—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://a10-bias-assessment-with-human-feedback.course-xai-iml23.isginf.ch/.

References

  1. Large-scale celebfaces attributes (celeba) dataset. https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html

  2. Baer, T.: Understand, manage, and prevent algorithmic bias: a guide for business users and data scientists. Apress, New York, NY (2019)

    Google Scholar 

  3. Balakrishnan, G., Xiong, Y., Xia, W., Perona, P.: Towards causal benchmarking of bias in face analysis algorithms. In: European Conference on Computer Vision (2020)

    Google Scholar 

  4. Bethlehem, J.: Selection bias in web surveys. Int. Statist. Rev. 78(2), 161–188 (2010). https://doi.org/10.1111/j.1751-5823.2010.00112.x

    Article  Google Scholar 

  5. Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on Fairness, Accountability and Transparency. PMLR (2018)

    Google Scholar 

  6. Cavazos, J.G., Phillips, P.J., Castillo, C.D., O’Toole, A.J.: Accuracy comparison across face recognition algorithms: where are we on measuring race bias? IEEE Trans. Biometrics, Behav. Identity Sci. 3(1), 101–111 (2021). https://doi.org/10.1109/TBIOM.2020.3027269

    Article  Google Scholar 

  7. Corbett-Davies, S., Gaebler, J., Nilforoshan, H., Shroff, R., Goel, S.: The measure and mismeasure of fairness. J. Mach. Learn. Res (2023)

    Google Scholar 

  8. De-Arteaga, M., et al.: Bias in bios: a case study of semantic representation bias in a high-stakes setting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency (2019)

    Google Scholar 

  9. Dulhanty, C., Wong, A.: Auditing ImageNet: towards a model-driven framework for annotating demographic attributes of large-scale image datasets. ArXiv (2019)

    Google Scholar 

  10. Fabbrizzi, S., Papadopoulos, S., Ntoutsi, E., Kompatsiaris, Y.: A survey on bias in visual datasets. Comput. Vis. Image Underst. 223 (2021)

    Google Scholar 

  11. F.R.S., K.P.: LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh Dublin Philos. Mag. J. Sci. 2(11) (1901)

    Google Scholar 

  12. Ghai, B., Mueller, K.: D-bias: a causality-based human-in-the-loop system for tackling algorithmic bias. IEEE Trans. Vis. Comput. Graph. (2022)

    Google Scholar 

  13. Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 27. Curran Associates, Inc. (2014)

    Google Scholar 

  14. Hu, X., et al.: Crowdsourcing detection of sampling biases in image datasets. In: Proceedings of The Web Conference 2020. WWW ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3366423.3380063

  15. Kärkkäinen, K., Joo, J.: FairFace: face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) (2019)

    Google Scholar 

  16. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  17. Kleinberg, J., Mullainathan, S., Raghavan, M.: Inherent Trade-Offs in the fair determination of risk scores. Conf. Innov. Theoret. Comput. Sci. 67, 23 (2017). https://doi.org/10.4230/LIPIcs.ITCS.2017.43

    Article  MathSciNet  Google Scholar 

  18. Koenecke, A., et al.: Racial disparities in automated speech recognition. Proc. Natl. Acad. Sci. 117(14) (2020)

    Google Scholar 

  19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6) (2017)

    Google Scholar 

  20. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  21. Martínez-Mesa, J., González-Chica, D.A., Duquia, R.P., Bonamigo, R.R., Bastos, J.L.: Sampling: how to select participants in my research study? An. Bras. Dermatol. 91, 326–330 (2016)

    Article  Google Scholar 

  22. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Comput. Surv. 54(6) (2021). https://doi.org/10.1145/3457607

  23. Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D.: Image segmentation using deep learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(7) (2021)

    Google Scholar 

  24. Model, I., Shamir, L.: Comparison of data set bias in object recognition benchmarks. IEEE Access 3 (2015)

    Google Scholar 

  25. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022). https://doi.org/10.1109/cvpr52688.2022.01042

  26. Rudd, E., Günther, M., Boult, T.: Moon: a mixed objective optimization network for the recognition of facial attributes, 9909 (2016). https://doi.org/10.1007/978-3-319-46454-1_2

  27. Singer, E., Ye, C.: The use and effects of incentives in surveys. Ann. Am. Acad. Polit. Soc. Sci. 645(1), 112–141 (2013). https://doi.org/10.1177/0002716212458082

    Article  Google Scholar 

  28. Surowiecki, J.: The Wisdom of Crowds. Anchor (2005)

    Google Scholar 

  29. Syakur, M., Khotimah, B., Rochman, E., Satoto, B.D.: Integration k-means clustering method and elbow method for identification of the best customer profile cluster. In: IOP Conference Series: Materials Science and Engineering, vol. 336. IOP Publishing (2018)

    Google Scholar 

  30. Taherdoost, H.: Sampling methods in research methodology; how to choose a sampling technique for research. How to choose a sampling technique for research (2016)

    Google Scholar 

  31. Thomas, C., Kovashka, A.: Predicting the politics of an image using webly supervised data. In: Advances in Neural Information Processing Systems 32 (2019)

    Google Scholar 

  32. Tommasi, T., Patricia, N., Caputo, B., Tuytelaars, T.: A deeper look at dataset bias (2017). https://doi.org/10.1007/978-3-319-58347-1_2

  33. Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: CVPR 2011 (2011)

    Google Scholar 

  34. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

  35. Verma, S., Rubin, J.: Fairness definitions explained. In: Proceedings of the International Workshop on Software Fairness (2018)

    Google Scholar 

  36. Wachinger, C., Rieckmann, A., Pölsterl, S.: Detect and correct bias in multi-site neuroimaging datasets. Med. Image Anal. 67 (2020)

    Google Scholar 

  37. Wang, Q., Xu, Z., Chen, Z., Wang, Y., Liu, S., Qu, H.: Visual analysis of discrimination in machine learning. IEEE Trans. Vis. Comput. Graph. 27, 1470–1480 (2020)

    Article  Google Scholar 

  38. Wang, T., Zhao, J., Yatskar, M., Chang, K.W., Ordonez, V.: Balanced datasets are not enough: estimating and mitigating gender bias in deep image representations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

    Google Scholar 

  39. Weizenbaum, J.: Eliza-a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1) (1966)

    Google Scholar 

  40. Xie, T., Ma, Y., Kang, J., Tong, H., Maciejewski, R.: FairRankVis: a visual analytics framework for exploring algorithmic fairness in graph mining models. IEEE Trans. Vis. Comput. Graph. (2022)

    Google Scholar 

  41. Yan, J.N., Gu, Z., Lin, H., Rzeszotarski, J.M.: Silva: interactively assessing machine learning fairness using causality. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. CHI ’20, Association for Computing Machinery, New York, NY, USA (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, ETH Zürich, Switzerland

    Senthuran Kalananthan, Ziyao Shang, András Strausz & Mennatallah El-Assady

  2. Department of Mathematics, ETH Zürich, Switzerland

    Alexander Kichutkin

  3. Department of Information Technologies and Electrical Engineering, ETH Zürich, Switzerland

    Francisco Javier Sanguino Bautiste

  4. AI Center, ETH Zürich, Switzerland

    Mennatallah El-Assady

Authors
  1. Senthuran Kalananthan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Kichutkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziyao Shang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. András Strausz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francisco Javier Sanguino Bautiste
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mennatallah El-Assady
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Kichutkin .

Editor information

Editors and Affiliations

  1. Halmstad University, Halmstad, Sweden

    Sławomir Nowaczyk

  2. Warsaw University of Technology, Warsaw, Poland

    Przemysław Biecek

  3. Warsaw University, Warsaw, Poland

    Neo Christopher Chung

  4. University of Huddersfield, Huddersfield, UK

    Mauro Vallati

  5. AGH University of Science and Technology, Kraków, Poland

    Paweł Skruch

  6. AGH University of Science and Technology, Kraków, Poland

    Joanna Jaworek-Korjakowska

  7. University of Huddersfield, Huddersfield, UK

    Simon Parkinson

  8. University of Huddersfield, Huddersfield, UK

    Alexandros Nikitas

  9. Universität Osnabrück, Osnabrück, Germany

    Martin Atzmüller

  10. University of Economics Prague, Prague, Czech Republic

    Tomáš Kliegr

  11. University of Bamberg, Bamberg, Germany

    Ute Schmid

  12. Jagiellonian University, Kraków, Poland

    Szymon Bobek

  13. Jožef Stefan Institute, Ljubljana, Slovenia

    Nada Lavrac

  14. HU University of Applied Sciences Utrecht, Utrecht, The Netherlands

    Marieke Peeters

  15. Rotterdam University of Applied Sciences, Rotterdam, The Netherlands

    Roland van Dierendonck

  16. Amsterdam University of Applied Sciences, Amsterdam, The Netherlands

    Saskia Robben

  17. University of Reims Champagne-Ardenne, Reims, France

    Eunika Mercier-Laurent

  18. Istanbul Technical University, Istanbul, Türkiye

    Gülgün Kayakutlu

  19. Wroclaw University of Economics and Business, Wrocław, Poland

    Mieczyslaw Lech Owoc

  20. University of Galway, Galway, Ireland

    Karl Mason

  21. University of Galway, Galway, Ireland

    Abdul Wahid

  22. University of Calabria, Rende, Italy

    Pierangela Bruno

  23. University of Calabria, Rende, Italy

    Francesco Calimeri

  24. Marche Polytechnic University, Ancona, Italy

    Francesco Cauteruccio

  25. University of Calabria, Rende, Italy

    Giorgio Terracina

  26. University of Bamberg, Bamberg, Germany

    Diedrich Wolter

  27. Coburg University of Applied Sciences, Coburg, Germany

    Jochen L. Leidner

  28. FAU Erlangen-Nürnberg, Erlangen, Germany

    Michael Kohlhase

  29. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kalananthan, S., Kichutkin, A., Shang, Z., Strausz, A., Bautiste, F.J.S., El-Assady, M. (2024). MindSet: A Bias-Detection Interface Using a Visual Human-in-the-Loop Workflow. In: Nowaczyk, S., et al. Artificial Intelligence. ECAI 2023 International Workshops. ECAI 2023. Communications in Computer and Information Science, vol 1948. Springer, Cham. https://doi.org/10.1007/978-3-031-50485-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-50485-3_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-50484-6

  • Online ISBN: 978-3-031-50485-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation