Abstract
Existing work on fairness modeling commonly assumes that sensitive attributes for all instances are fully available, which may not be true in many real-world applications due to the high cost of acquiring sensitive information. When sensitive attributes are not disclosed or available, it is needed to manually annotate a small part of the training data to mitigate bias. However, the skewed distribution across different sensitive groups preserves the skewness of the original dataset in the annotated subset, which leads to non-optimal bias mitigation. To tackle this challenge, we propose Active Penalization Of Discrimination (APOD), an interactive framework to guide the limited annotations towards maximally eliminating the effect of algorithmic bias. The proposed APOD integrates discrimination penalization with active instance selection to efficiently utilize the limited annotation budget, and it is theoretically proved to be capable of bounding the algorithmic bias. According to the evaluation on five benchmark datasets, APOD outperforms the state-of-the-arts baseline methods under the limited annotation budget, and shows comparable performance to fully annotated bias mitigation, which demonstrates that APOD could benefit real-world applications when sensitive information is limited. The source code of the proposed method is available at: https://github.com/guanchuwang/APOD-fairness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The combination of TPR and FPR is representative enough accross different fairness metrics. POD is flexible to use other metrics as the regularizer for the bias mitigation.
- 2.
It also has other choices for the relaxation, e.g. sigmoid and tanh functions. The linear function is chosen for simplicity.
- 3.
The training error is less than generalization error in most cases.
- 4.
\(\epsilon \) can be very small if the classifier head \(f_h\) has been well-trained on the annotated dataset \(\mathcal {S}\).
- 5.
\(l(\boldsymbol{h}, y; \theta _h)\) and \(f_h\) satisfy \(|l(\boldsymbol{h}_i, y; \theta _h) - l(\boldsymbol{h}_j, y; \theta _h)| \le K_l ||\boldsymbol{h}_{i} - \boldsymbol{h}_{j}||_2\) and \(|p(y | \boldsymbol{x}_i) - p(y | \boldsymbol{x}_j)| \le K_h ||\boldsymbol{h}_{i} - \boldsymbol{h}_{j}||_2\), respectively, where the likelihood function \(p(y \mid \boldsymbol{x}_i) = \text {softmax}(f_h(\boldsymbol{h}_i | \theta _h))\).
References
Abernethy, J.D., Awasthi, P., Kleindessner, M., Morgenstern, J., Russell, C., Zhang, J.: Active sampling for min-max fairness. In: International Conference on Machine Learning. vol. 162 (2022)
Anahideh, H., Asudeh, A., Thirumuruganathan, S.: Fair active learning. arXiv preprint arXiv:2001.01796 (2020)
Angwin, J., Larson, J., Mattu, S., Kirchner, L.: There’s software used across the country to predict future criminals. ProPublica (2016)
Azzalini, A.: The skew-normal distribution and related multivariate families. Scand. J. Stat. 32(2), 159–188 (2005)
Bechavod, Y., Ligett, K.: Penalizing unfairness in binary classification. arXiv preprint arXiv:1707.00044 (2017)
Caton, S., Haas, C.: Fairness in machine learning: a survey. arXiv preprint arXiv:2010.04053 (2020)
Chai, J., Jang, T., Wang, X.: Fairness without demographics through knowledge distillation. In: Advances in Neural Information Processing Systems
Chai, J., Wang, X.: Self-supervised fair representation learning without demographics. In: Advances in Neural Information Processing Systems
Chuang, C.Y., Mroueh, Y.: Fair mixup: Fairness via interpolation. arXiv preprint arXiv:2103.06503 (2021)
Chuang, Y.N., et al.: Mitigating relational bias on knowledge graphs. arXiv preprint arXiv:2211.14489 (2022)
Chuang, Y.N., et al.: Efficient XAI techniques: A taxonomic survey. arXiv preprint arXiv:2302.03225 (2023)
Chuang, Y.N., et al.: CoRTX: Contrastive framework for real-time explanation. arXiv preprint arXiv:2303.02794 (2023)
Creager, E., et al.: Flexibly fair representation learning by disentanglement. In: International Conference on Machine Learning, pp. 1436–1445. PMLR (2019)
Dai, E., Wang, S.: Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 680–688 (2021)
Deng, Z., et al.: FIFA: Making fairness more generalizable in classifiers trained on imbalanced data. arXiv preprint arXiv:2206.02792 (2022)
Du, M., Mukherjee, S., Wang, G., Tang, R., Awadallah, A., Hu, X.: Fairness via representation neutralization. In: Advances in Neural Information Processing Systems. vol. 34 (2021)
Du, M., Yang, F., Zou, N., Hu, X.: Fairness in deep learning: a computational perspective. IEEE Intell. Syst. 36(4), 25–34 (2020)
Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
Goel, S., Rao, J.M., Shroff, R., et al.: Precinct or prejudice? Understanding racial disparities in New York city’s stop-and-frisk policy. Ann. Appl. Stat. 10(1), 365–394 (2016)
Han, X., et al.: Retiring \(\delta \text{DP}\): New distribution-level metrics for demographic parity. Transactions on Machine Learning Research (2023). https://openreview.net/forum?id=LjDFIWWVVa
Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. Adv. Neural Inf. Process. Syst. 29, 3315–3323 (2016)
Hashimoto, T., Srivastava, M., Namkoong, H., Liang, P.: Fairness without demographics in repeated loss minimization. In: International Conference on Machine Learning, pp. 1929–1938. PMLR (2018)
Jiang, Z., et al.: FMP: Toward fair graph message passing against topology bias. arXiv preprint arXiv:2202.04187 (2022)
Jiang, Z., Han, X., Fan, C., Yang, F., Mostafavi, A., Hu, X.: Generalized demographic parity for group fairness. In: International Conference on Learning Representations (2022)
Jiang, Z., Han, X., Jin, H., Wang, G., Zou, N., Hu, X.: Weight perturbation can help fairness under distribution shift. arXiv preprint arXiv:2303.03300 (2023)
Kallus, N., Mao, X., Zhou, A.: Assessing algorithmic fairness with unobserved protected class using data combination. Manag. Sci. 68(3), 1591–2376 (2021)
Kleinberg, J., Ludwig, J., Mullainathan, S., Rambachan, A.: Algorithmic fairness. In: Aea Papers and Proceedings. vol. 108, pp. 22–27 (2018)
Lahoti, P., et al.: Fairness without demographics through adversarially reweighted learning. arXiv preprint arXiv:2006.13114 (2020)
Li, Y., Vasconcelos, N.: Repair: Removing representation bias by dataset resampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9572–9581 (2019)
Ling, H., Jiang, Z., Luo, Y., Ji, S., Zou, N.: Learning fair graph representations via automated data augmentations. In: International Conference on Learning Representations (2023)
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Comput. Surv. (CSUR) 54(6), 1–35 (2021)
Mehrotra, A., Vishnoi, N.K.: Fair ranking with noisy protected attributes. In: Advances in Neural Information Processing Systems
Nam, J., Cha, H., Ahn, S., Lee, J., Shin, J.: Learning from failure: Training debiased classifier from biased classifier. arXiv preprint arXiv:2007.02561 (2020)
Romano, Y., Bates, S., Candes, E.J.: Achieving equalized odds by resampling sensitive attributes. arXiv preprint arXiv:2006.04292 (2020)
Sagawa, S., Koh, P.W., Hashimoto, T.B., Liang, P.: Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 (2019)
Sener, O., Savarese, S.: Active learning for convolutional neural networks: A core-set approach (2018)
Slack, D., Friedler, S.A., Givental, E.: Fairness warnings and fair-MAML: learning fairly with minimal data. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 200–209 (2020)
Steel, E., Angwin, J.: On the web’s cutting edge, anonymity in name only. The Wall Street Journal 4 (2010)
Sun, T., et al.: Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976 (2019)
Sweeney, L.: Discrimination in online ad delivery. Commun. ACM 56(5), 44–54 (2013)
Tang, R., Du, M., Li, Y., Liu, Z., Zou, N., Hu, X.: Mitigating gender bias in captioning systems. In: Proceedings of the Web Conference 2021, pp. 633–645 (2021)
Verma, S., Rubin, J.: Fairness definitions explained. In: 2018 IEEE/ACM International Workshop on Software Fairness (fairware), pp. 1–7. IEEE (2018)
Wang, G., et al.: Accelerating shapley explanation via contributive cooperator selection. In: International Conference on Machine Learning, pp. 22576–22590. PMLR (2022)
Wang, S., Guo, W., Narasimhan, H., Cotter, A., Gupta, M., Jordan, M.I.: Robust optimization for fairness with noisy protected groups. arXiv preprint arXiv:2002.09343 (2020)
Zha, D., et al.: Data-centric artificial intelligence: A survey. arXiv preprint arXiv:2303.10158 (2023)
Zhang, B.H., Lemoine, B., Mitchell, M.: Mitigating unwanted biases with adversarial learning. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 335–340 (2018)
Zhang, F., Kuang, K., Chen, L., Liu, Y., Wu, C., Xiao, J.: Fairness-aware contrastive learning with partially annotated sensitive attributes. In: The Eleventh International Conference on Learning Representations
Zhao, T., Dai, E., Shu, K., Wang, S.: You can still achieve fairness without sensitive attributes: Exploring biases in non-sensitive features. arXiv preprint arXiv:2104.14537 (2021)
Zimdars, A.: Fairness and undergraduate admission: a qualitative exploration of admissions choices at the university of Oxford. Oxford Rev. Educ. 36(3), 307–323 (2010)
Acknowledgement
The authors thank the anonymous reviewers for their helpful comments. The work is in part supported by NSF grants NSF IIS-1939716, IIS-1900990, and IIS-2239257. The views and conclusions contained in this paper are those of the authors and should not be interpreted as representing any funding agencies.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Ethical Statement
This paper has been thoroughly reviewed for ethical considerations and has been found to be in compliance with all relevant ethical guidelines. The paper does not raise any ethical concerns and is a valuable contribution to the field.
Appendix
Appendix
The appendix is available at https://github.com/guanchuwang/APOD-fairness/blob/main/appendix/bias_mitigation_appendix.pdf.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, G., Du, M., Liu, N., Zou, N., Hu, X. (2023). Mitigating Algorithmic Bias with Limited Annotations. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14170. Springer, Cham. https://doi.org/10.1007/978-3-031-43415-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-43415-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43414-3
Online ISBN: 978-3-031-43415-0
eBook Packages: Computer ScienceComputer Science (R0)
