Skip to main content

Discover and Mitigate Unknown Biases with Debiasing Alternate Networks

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13673))

Included in the following conference series:

Abstract

Deep image classifiers have been found to learn biases from datasets. To mitigate the biases, most previous methods require labels of protected attributes (e.g., age, skin tone) as full-supervision, which has two limitations: 1) it is infeasible when the labels are unavailable; 2) they are incapable of mitigating unknown biases—biases that humans do not preconceive. To resolve those problems, we propose Debiasing Alternate Networks (DebiAN), which comprises two networks—a Discoverer and a Classifier. By training in an alternate manner, the discoverer tries to find multiple unknown biases of the classifier without any annotations of biases, and the classifier aims at unlearning the biases identified by the discoverer. While previous works evaluate debiasing results in terms of a single bias, we create Multi-Color MNIST dataset to better benchmark mitigation of multiple biases in a multi-bias setting, which not only reveals the problems in previous methods but also demonstrates the advantage of DebiAN in identifying and mitigating multiple biases simultaneously. We further conduct extensive experiments on real-world datasets, showing that the discoverer in DebiAN can identify unknown biases that may be hard to be found by humans. Regarding debiasing, DebiAN achieves strong bias mitigation performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    In this work, “gender” denotes visually perceived gender, not real gender identity.

References

  1. Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: International Conference on Very Large Data Bases (1994)

    Google Scholar 

  2. Ahmed, F., Bengio, Y., van Seijen, H., Courville, A.: Systematic generalisation with group invariant predictions. In: International Conference on Learning Representations (2021)

    Google Scholar 

  3. Albiero, V., KS, K., Vangara, K., Zhang, K., King, M.C., Bowyer, K.W.: Analysis of gender inequality in face recognition accuracy. In: The IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW) (2020)

    Google Scholar 

  4. Alvi, M., Zisserman, A., Nellaaker, C.: Turning a blind eye: explicit removal of biases and variation from deep neural network embeddings. In: The European Conference on Computer Vision Workshop (ECCVW) (2018)

    Google Scholar 

  5. Antol, S., et al.: VQA: visual question answering. In: The IEEE International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  6. Arjovsky, M., Bottou, L., Gulrajani, I., Lopez-Paz, D.: invariant risk minimization. arXiv:1907.02893 [cs, stat] (2020)

  7. Bahng, H., Chun, S., Yun, S., Choo, J., Oh, S.J.: Learning de-biased representations with biased representations. In: International Conference on Machine Learning (2020)

    Google Scholar 

  8. Balakrishnan, G., Xiong, Y., Xia, W., Perona, P.: Towards causal benchmarking of bias in face analysis algorithms. In: The European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

  9. Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. In: ACM Conference on Fairness, Accountability, and Transparency (2018)

    Google Scholar 

  10. Cadene, R., Dancette, C., Ben younes, H., Cord, M., Parikh, D.: RUBi: reducing unimodal biases for visual question answering. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  11. Choi, J., Gao, C., Messou, J.C.E., Huang, J.B.: Why can’t i dance in the mall? Learning to mitigate scene bias in action recognition. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  12. Clark, C., Yatskar, M., Zettlemoyer, L.: don’t take the easy way out: ensemble based methods for avoiding known dataset biases. In: Empirical Methods in Natural Language Processing (2019)

    Google Scholar 

  13. Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2017)

    Google Scholar 

  14. Creager, E., Jacobsen, J.H., Zemel, R.: Environment inference for invariant learning. In: International Conference on Machine Learning (2021)

    Google Scholar 

  15. Creager, E., et al.: Flexibly fair representation learning by disentanglement. In: International Conference on Machine Learning (2019)

    Google Scholar 

  16. Dhar, P., Gleason, J., Roy, A., Castillo, C.D., Chellappa, R.: PASS: protected attribute suppression system for mitigating bias in face recognition. In: The IEEE International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

  17. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (2012)

    Google Scholar 

  18. Geirhos, R., et al.: Shortcut learning in deep neural networks. Nat. Mach. Intell. 2(11), 665–673 (2020)

    Google Scholar 

  19. Gong, S., Liu, X., Jain, A.K.: Jointly de-biasing face recognition and demographic attribute estimation. In: The European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

  20. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (2014)

    Google Scholar 

  21. Grgic-Hlaca, N., Zafar, M.B., Gummadi, K.P., Weller, A.: The case for process fairness in learning: Feature selection for fair decision making. In: NIPS Symposium on Machine Learning and the Law (2016)

    Google Scholar 

  22. Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. In: Advances in Neural Information Processing Systems (2016)

    Google Scholar 

  23. Hazirbas, C., Bitton, J., Dolhansky, B., Pan, J., Gordo, A., Ferrer, C.C.: Towards measuring fairness in AI: the casual conversations dataset. arXiv:2104.02821 [cs] (2021)

  24. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  25. Hendricks, L.A., Burns, K., Saenko, K., Darrell, T., Rohrbach, A.: Women also snowboard: overcoming bias in captioning models. In: The European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  26. Jia, S., Meng, T., Zhao, J., Chang, K.W.: Mitigating gender bias amplification in distribution by posterior regularization. In: Annual Meeting of the Association for Computational Linguistics (2020)

    Google Scholar 

  27. Joo, J., Kärkkäinen, K.: Gender slopes: counterfactual fairness for computer vision models by attribute manipulation. In: International Workshop on Fairness, Accountability, Transparency and Ethics in Multimedia (2020)

    Google Scholar 

  28. Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33, 1–33 (2012). https://doi.org/10.1007/s10115-011-0463-8

  29. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  30. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  31. Kim, B., Kim, H., Kim, K., Kim, S., Kim, J.: Learning not to learn: training deep neural networks with biased data. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  32. Kim, E., Lee, J., Choo, J.: BiaSwap: removing dataset bias with bias-tailored swapping augmentation. In: The IEEE International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

  33. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)

    Google Scholar 

  34. Krishnakumar, A., Prabhu, V., Sudhakar, S., Hoffman, J.: UDIS: unsupervised discovery of bias in deep visual recognition models. In: British Machine Vision Conference, BMVC (2021)

    Google Scholar 

  35. Kusner, M.J., Loftus, J., Russell, C., Silva, R.: Counterfactual fairness. In: Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  36. Lahoti, P., et al.: Fairness without demographics through adversarially reweighted learning. In: Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  37. Lang, O., et al.: Explaining in style: training a GAN to explain a classifier in StyleSpace. In: The IEEE International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

  38. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE (1998)

    Google Scholar 

  39. Li, W., et al.: Object-driven text-to-image synthesis via adversarial training. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  40. Li, Y., Li, Y., Vasconcelos, N.: RESOUND: towards action recognition without Representation Bias. In: The European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  41. Li, Y., Vasconcelos, N.: REPAIR: removing representation bias by dataset resampling. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  42. Li, Z., Xu, C.: Discover the unknown biased attribute of an image classifier. In: The IEEE International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

  43. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: The IEEE International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  44. Manjunatha, V., Saini, N., Davis, L.S.: Explicit bias discovery in visual question answering models. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  45. Nam, J., Cha, H., Ahn, S., Lee, J., Shin, J.: Learning from failure: training debiased classifier from biased classifier. In: Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  46. Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., Weinberger, K.Q.: On fairness and calibration. In: Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  47. Sagawa*, S., Koh*, P.W., Hashimoto, T.B., Liang, P.: Distributionally robust neural networks for group shifts: on the importance of regularization for worst-case generalization. In: International Conference on Learning Representations (2020)

    Google Scholar 

  48. Sarhan, M.H., Navab, N., Albarqouni, S.: Fairness by learning orthogonal disentangled representations. In: The European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

  49. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: The IEEE International Conference on Computer Vision (ICCV) (2017)

    Google Scholar 

  50. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128(2), 336–359 (2020). https://doi.org/10.1007/s11263-019-01228-7

    Article  Google Scholar 

  51. Singh, K.K., Mahajan, D., Grauman, K., Lee, Y.J., Feiszli, M., Ghadiyaram, D.: Don’t judge an object by its context: learning to overcome contextual bias. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  52. Sohoni, N.S., Dunnmon, J.A., Angus, G., Gu, A., Ré, C.: No subclass left behind: fine-grained robustness in coarse-grained classification problems. In: Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  53. Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)

    Google Scholar 

  54. Tzeng, E., Hoffman, J., Darrell, T., Saenko, K.: Simultaneous deep transfer across domains and tasks. In: The IEEE International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  55. Verma, S., Rubin, J.: Fairness definitions explained. In: 2018 IEEE/ACM International Workshop on Software Fairness (FairWare) (2018)

    Google Scholar 

  56. Wang, A., Narayanan, A., Russakovsky, O.: REVISE: a tool for measuring and mitigating bias in image datasets. In: The European Conference on Computer Vision (ECCV) (2020a)

    Google Scholar 

  57. Wang, H., He, Z., Lipton, Z.C., Xing, E.P.: Learning robust representations by projecting superficial statistics out. In: International Conference on Learning Representations (2019a)

    Google Scholar 

  58. Wang, J., Liu, Y., Wang, X.E.: Are gender-neutral queries really gender-neutral? mitigating gender bias in image search. In: Empirical Methods in Natural Language Processing (2021a)

    Google Scholar 

  59. Wang, T., Yue, Z., Huang, J., Sun, Q., Zhang, H.: Self-supervised learning disentangled group representation as feature. In: Advances in Neural Information Processing Systems (2021b)

    Google Scholar 

  60. Wang, X., Ang, M.H., Lee, G.H.: Cascaded refinement network for point cloud completion. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020b)

    Google Scholar 

  61. Wang, Z., et al.: CAMP: cross-modal adaptive message passing for text-image retrieval. In: The IEEE International Conference on Computer Vision (ICCV) (2019b)

    Google Scholar 

  62. Wang, Z., Qinami, K., Karakozis, I.C., Genova, K., Nair, P., Hata, K., Russakovsky, O.: Towards fairness in visual recognition: effective strategies for bias mitigation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020c)

    Google Scholar 

  63. Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., Xiao, J.: LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv:1506.03365 [cs] (2016)

  64. Zhang, B.H., Lemoine, B., Mitchell, M.: Mitigating unwanted biases with adversarial learning. In: AAAI/ACM Conference on AI, Ethics, and Society (2018)

    Google Scholar 

  65. Zhang, Z., Sabuncu, M.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Advances in Neural Information Processing Systems (2018)

    Google Scholar 

  66. Zhao, J., Wang, T., Yatskar, M., Ordonez, V., Chang, K.W.: Men also like shopping: reducing gender bias amplification using corpus-level constraints. In: Empirical Methods in Natural Language Processing (2017)

    Google Scholar 

  67. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  68. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018)

    Article  Google Scholar 

Download references

Acknowledgement

This work has been partially supported by the National Science Foundation (NSF) under Grant 1764415, 1909912, and 1934962 and by the Center of Excellence in Data Science, an Empire State Development-designated Center of Excellence. The article solely reflects the opinions and conclusions of its authors but not the funding agents.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chenliang Xu .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1355 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Z., Hoogs, A., Xu, C. (2022). Discover and Mitigate Unknown Biases with Debiasing Alternate Networks. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13673. Springer, Cham. https://doi.org/10.1007/978-3-031-19778-9_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19778-9_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19777-2

  • Online ISBN: 978-3-031-19778-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics