Abstract
While conventional ranking systems focus solely on maximizing the utility of the ranked items to users, fairness-aware ranking systems additionally try to balance the exposure based on different protected attributes such as gender or race. To achieve this type of group fairness for ranking, we derive a new ranking system from the first principles of distributional robustness. We formulate a minimax game between a player choosing a distribution over rankings to maximize utility while satisfying fairness constraints against an adversary seeking to minimize utility while matching statistics of the training data. Rather than maximizing utility and fairness for the specific training data, this approach efficiently produces robust utility and fairness for a much broader family of distributions of rankings that include the training data. We show that our approach provides better utility for highly fair rankings than existing baseline methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We use the implementation from https://github.com/ashudeep/Fair-PGRank for all baselines.
References
Beutel, A., et al.: Fairness in recommendation ranking through pairwise comparisons. In: ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(KDD), pp. 2212–2220 (2019)
Biega, A.J., Gummadi, K.P., Weikum, G.: Equity of attention: Amortizing individual fairness in rankings. In: The 41st International ACM Sigir Conference on Research and Development in Information Retrieval, pp. 405–414 (2018)
Boyd, S., Parikh, N., Chu, E.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc (2011)
Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190–1208 (1995)
Cao, Z., Qin, T., Liu, T.Y., Tsai, M.F., Li, H.: Learning to rank: from pairwise approach to listwise approach. In: Proceedings of the 24th International Conference on Machine Learning, pp. 129–136 (2007)
Celis, L.E., Mehrotra, A., Vishnoi, N.K.: Interventions for ranking in the presence of implicit bias. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 369–380 (2020)
Deng, W., Yin, W.: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3) (2016)
Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
Diaz, F., Mitra, B., Ekstrand, M.D., Biega, A.J., Carterette, B.: Evaluating stochastic rankings with expected exposure. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 275–284 (2020)
Eckhouse, L.: Big data may be reinforcing racial bias in the criminal justice system. The Washington Post (2017)
Fathony, R., Liu, A., Asif, K., Ziebart, B.: Adversarial multiclass classification: A risk minimization perspective. In: NeurIPS (2016)
Ghosh, A., Dutt, R., Wilson, C.: When fair ranking meets uncertain inference. arXiv preprint arXiv:2105.02091 (2021)
Grünwald, P.D., Dawid, A.P.: Game theory, maximum entropy, minimum discrepancy, and robust Bayesian decision theory. Ann. Stat. 32, 1367–1433 (2004)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of ir techniques. ACM Trans. Inform. Syst. (TOIS) 20(4), 422–446 (2002)
Kallus, N., Zhou, A.: The fairness of risk scores beyond classification: Bipartite ranking and the xauc metric. In: Advances in Neural Information Processing Systems, pp. 3438–3448 (2019)
Kleinberg, J., Raghavan, M.: Selection problems in the presence of implicit bias. In: 9th Innovations in Theoretical Computer Science Conference (ITCS 2018) (2018)
Kuhn, H.W.: The hungarian method for the assignment problem. Naval Res. Logist. Quart. 2(1–2), 83–97 (1955)
Larson, J., Mattu, S., Kirchner, L., Angwin, J.: How we analyzed the compas recidivism algorithm. ProPublica 9 (2016)
Mehrotra, A., Celis, L.E.: Mitigating bias in set selection with noisy protected attributes. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 237–248 (2021)
Narasimhan, H., Cotter, A., Gupta, M., Wang, S.: Pairwise fairness for ranking and regression. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 5248–5255 (2020)
O’Neil, C.: Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books (2016)
Prost, F., et al.: Measuring model fairness under noisy covariates: A theoretical perspective. arXiv preprint arXiv:2105.09985 (2021)
Qin, T., Liu, T.Y.: Introducing letor 4.0 datasets. arXiv preprint arXiv:1306.2597 (2013)
Rezaei, A., Fathony, R., Memarrast, O., Ziebart, B.: Fairness for robust log loss classification. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 5511–5518 (2020)
Rezaei, A., Liu, A., Memarrast, O., Ziebart, B.D.: Robust fairness under covariate shift. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 9419–9427 (2021)
Robertson, S.E.: The probability ranking principle in ir. J. Document. 33(4), 294–304 (1977)
Singh, A., Joachims, T.: Fairness of exposure in rankings. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2219–2228. ACM (2018)
Singh, A., Joachims, T.: Policy learning for fairness in ranking. Adv. Neural. Inf. Process. Syst. 32, 5426–5436 (2019)
Singh, A., Kempe, D., Joachims, T.: Fairness in ranking under uncertainty. In: Advances in Neural Information Processing Systems, p. 34 (2021)
Yadav, H., Du, Z., Joachims, T.: Policy-gradient training of fair and unbiased ranking functions. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1044–1053 (2021)
Zehlike, M., Castillo, C.: Reducing disparate exposure in ranking: A learning to rank approach. In: Proceedings of The Web Conference 2020, pp. 2849–2855 (2020)
Acknowledgements
This work was supported by the National Science Foundation Program on Fairness in AI in collaboration with Amazon under award No. 1939743.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Memarrast, O., Rezaei, A., Fathony, R., Ziebart, B. (2023). Fairness for Robust Learning to Rank. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13935. Springer, Cham. https://doi.org/10.1007/978-3-031-33374-3_43
Download citation
DOI: https://doi.org/10.1007/978-3-031-33374-3_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33373-6
Online ISBN: 978-3-031-33374-3
eBook Packages: Computer ScienceComputer Science (R0)