Skip to main content
SpringerLink
Log in

How to Forget Clients in Federated Online Learning to Rank?

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14610))

Included in the following conference series:

  • 452 Accesses

Abstract

Data protection legislation like the European Union’s General Data Protection Regulation (GDPR) establishes the right to be forgotten: a user (client) can request contributions made using their data to be removed from learned models. In this paper, we study how to remove the contributions made by a client participating in a Federated Online Learning to Rank (FOLTR) system. In a FOLTR system, a ranker is learned by aggregating local updates to the global ranking model. Local updates are learned in an online manner at a client-level using queries and implicit interactions that have occurred within that specific client. By doing so, each client’s local data is not shared with other clients or with a centralised search service, while at the same time clients can benefit from an effective global ranking model learned from contributions of each client in the federation.

In this paper, we study an effective and efficient unlearning method that can remove a client’s contribution without compromising the overall ranker effectiveness and without needing to retrain the global ranker from scratch. A key challenge is how to measure whether the model has unlearned the contributions from the client \(c^*\) that has requested removal. For this, we instruct \(c^*\) to perform a poisoning attack (add noise to this client updates) and then we measure whether the impact of the attack is lessened when the unlearning process has taken place. Through experiments on four datasets, we demonstrate the effectiveness and efficiency of the unlearning strategy under different combinations of parameter settings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Similar concepts as cross-device and cross-silo Federated Learning [21].

  2. 2.

    For example, many methods that are successful for dealing with non identical and independently distributed data (non-IID) in general federated learning do not work in FOLTR [43].

  3. 3.

    In federated learning, the local model can be trained on the local data repeatedly across several epochs, while in FOLTR, training data is acquired in real time as user interactions occur and it cannot be repeated and reused (e.g., a user cannot be asked to submit the same query they did in the past, and perform the same interactions).

  4. 4.

    In our empirical study, we adapt FPDGD in which PDGD algorithm is used in the local training phase. Detailed method is specified in the original paper [41].

  5. 5.

    These local updates would ideally be stored within each client, but they could be stored instead in the central server: this though would required extra communication cost to provide the local updates back to the clients when needed.

  6. 6.

    The effectiveness of 9H-1M is not shown in Fig. 3 for clarity. The reader can cross reference Fig. 3 with Fig. 2, which instead contains the effectiveness of 9H-1M.

References

  1. Ai, Q., Yang, T., Wang, H., Mao, J.: Unbiased learning to rank: online or offline? ACM Trans. Inf. Syst. (TOIS) 39(2), 1–29 (2021)

    Article  Google Scholar 

  2. Aldaghri, N., Mahdavifar, H., Beirami, A.: Coded machine unlearning. IEEE Access 9, 88137–88150 (2021)

    Article  Google Scholar 

  3. Baruch, G., Baruch, M., Goldberg, Y.: A little is enough: circumventing defenses for distributed learning. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  4. Baumhauer, T., Schöttle, P., Zeppelzauer, M.: Machine unlearning: linear filtration for logit-based classifiers. Mach. Learn. 111(9), 3203–3226 (2022)

    Article  MathSciNet  Google Scholar 

  5. Bourtoule, L., et al.: Machine unlearning. In: 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159. IEEE (2021)

    Google Scholar 

  6. Brophy, J., Lowd, D.: Machine unlearning for random forests. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18–24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp. 1092–1104. PMLR (2021). https://proceedings.mlr.press/v139/brophy21a.html

  7. Cao, Y., Yang, J.: Towards making systems forget with machine unlearning. In: 2015 IEEE Symposium on Security and Privacy, SP 2015, San Jose, CA, USA, 17–21 May 2015, pp. 463–480. IEEE Computer Society (2015). https://doi.org/10.1109/SP.2015.35

  8. Chapelle, O., Chang, Y.: Yahoo! Learning to rank challenge overview. In: Proceedings of the Learning to Rank Challenge, pp. 1–24. PMLR (2011)

    Google Scholar 

  9. Che, T., et al.: Fast federated machine unlearning with nonlinear functional theory. In: International Conference on Machine Learning, pp. 4241–4268. PMLR (2023)

    Google Scholar 

  10. Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., Zhang, Y.: When machine unlearning jeopardizes privacy. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 896–911 (2021)

    Google Scholar 

  11. Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., Zhang, Y.: Graph unlearning. In: Yin, H., Stavrou, A., Cremers, C., Shi, E. (eds.) Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, CCS 2022, Los Angeles, CA, USA, 7–11 November 2022, pp. 499–513. ACM (2022). https://doi.org/10.1145/3548606.3559352

  12. Chen, Y., Xiong, J., Xu, W., Zuo, J.: A novel online incremental and decremental learning algorithm based on variable support vector machine. Clust. Comput. 22(Supplement), 7435–7445 (2019)

    Google Scholar 

  13. Chundawat, V.S., Tarun, A.K., Mandal, M., Kankanhalli, M.: Zero-shot machine unlearning. IEEE Trans. Inf. Forensics Secur. 18, 2345–2354 (2023)

    Article  Google Scholar 

  14. Gao, X., et al.: VeriFi: towards verifiable federated unlearning. arXiv preprint arXiv:2205.12709 (2022)

  15. Ginart, A., Guan, M.Y., Valiant, G., Zou, J.: Making AI forget you: data deletion in machine learning. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, Vancouver, BC, Canada, 8–14 December 2019, pp. 3513–3526 (2019). https://proceedings.neurips.cc/paper/2019/hash/cb79f8fa58b91d3af6c9c991f63962d3-Abstract.html

  16. Guo, F., Liu, C., Wang, Y.M.: Efficient multiple-click models in web search. In: Proceedings of the second ACM International Conference on Web Search and Data Mining, pp. 124–131 (2009)

    Google Scholar 

  17. Halimi, A., Kadhe, S., Rawat, A., Baracaldo, N.: Federated unlearning: how to efficiently erase a client in FL? arXiv preprint arXiv:2207.05521 (2022)

  18. Hofmann, K., Schuth, A., Whiteson, S., De Rijke, M.: Reusing historical interaction data for faster online learning to rank for IR. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pp. 183–192 (2013)

    Google Scholar 

  19. Hu, H., Salcic, Z., Dobbie, G., Chen, J., Sun, L., Zhang, X.: Membership inference via backdooring. In: The 31st International Joint Conference on Artificial Intelligence (IJCAI 2022) (2022)

    Google Scholar 

  20. Jia, Y., Wang, H.: Learning neural ranking models online from implicit user feedback. In: Proceedings of the ACM Web Conference 2022, pp. 431–441 (2022)

    Google Scholar 

  21. Kairouz, P., et al.: Advances and open problems in federated learning. Found. Trends® Mach. Learn. 14(1–2), 1–210 (2021)

    Google Scholar 

  22. Kharitonov, E.: Federated online learning to rank with evolution strategies. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 249–257 (2019)

    Google Scholar 

  23. Liu, G., Ma, X., Yang, Y., Wang, C., Liu, J.: FedEraser: enabling efficient client-level data removal from federated learning models. In: 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS), pp. 1–10. IEEE (2021)

    Google Scholar 

  24. Liu, Y., Ma, Z., Liu, X., Ma, J.: Learn to forget: user-level memorization elimination in federated learning. CoRR abs/2003.10933 (2020). https://arxiv.org/abs/2003.10933

  25. Liu, Y., Xu, L., Yuan, X., Wang, C., Li, B.: The right to be forgotten in federated learning: an efficient realization with rapid retraining. In: IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, London, United Kingdom, 2–5 May 2022, pp. 1749–1758. IEEE (2022). https://doi.org/10.1109/INFOCOM48880.2022.9796721

  26. Lucchese, C., Nardini, F.M., Orlando, S., Perego, R., Silvestri, F., Trani, S.: Post-learning optimization of tree ensembles for efficient ranking. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 949–952 (2016)

    Google Scholar 

  27. de Magalhães, S.T.: The European union’s general data protection regulation (GDPR). In: Cyber Security Practitioner’s Guide, pp. 529–558. World Scientific (2020)

    Google Scholar 

  28. Mahadevan, A., Mathioudakis, M.: Certifiable machine unlearning for linear models. arXiv preprint arXiv:2106.15093 (2021)

  29. McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)

    Google Scholar 

  30. Nguyen, T.T., Huynh, T.T., Nguyen, P.L., Liew, A.W.C., Yin, H., Nguyen, Q.V.H.: A survey of machine unlearning. arXiv preprint arXiv:2209.02299 (2022)

  31. Oosterhuis, H., de Rijke, M.: Differentiable unbiased online learning to rank. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 1293–1302 (2018)

    Google Scholar 

  32. Qin, T., Liu, T.Y.: Introducing LETOR 4.0 datasets. arXiv preprint arXiv:1306.2597 (2013)

  33. Salimans, T., Ho, J., Chen, X., Sidor, S., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864 (2017)

  34. Schuth, A., Oosterhuis, H., Whiteson, S., de Rijke, M.: Multileave gradient descent for fast online learning to rank. In: Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pp. 457–466 (2016)

    Google Scholar 

  35. Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18. IEEE (2017)

    Google Scholar 

  36. Sommer, D.M., Song, L., Wagh, S., Mittal, P.: Towards probabilistic verification of machine unlearning. arXiv preprint arXiv:2003.04247 (2020)

  37. Sommer, D.M., Song, L., Wagh, S., Mittal, P.: Athena: probabilistic verification of machine unlearning. Proc. Priv. Enhancing Technol. 3, 268–290 (2022)

    Article  Google Scholar 

  38. Wang, H., Kim, S., McCord-Snook, E., Wu, Q., Wang, H.: Variance reduction in gradient exploration for online learning to rank. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 835–844 (2019)

    Google Scholar 

  39. Wang, H., Langley, R., Kim, S., McCord-Snook, E., Wang, H.: Efficient exploration of gradient space for online learning to rank. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 145–154 (2018)

    Google Scholar 

  40. Wang, J., Guo, S., Xie, X., Qi, H.: Federated unlearning via class-discriminative pruning. In: Proceedings of the ACM Web Conference 2022, pp. 622–632 (2022)

    Google Scholar 

  41. Wang, S., Liu, B., Zhuang, S., Zuccon, G.: Effective and privacy-preserving federated online learning to rank. In: Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval, pp. 3–12 (2021)

    Google Scholar 

  42. Wang, S., Zhuang, S., Zuccon, G.: Federated online learning to rank with evolution strategies: a reproducibility study. In: European Conference on Information Retrieval (2021)

    Google Scholar 

  43. Wang, S., Zuccon, G.: Is non-IID data a threat in federated online learning to rank? In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2801–2813 (2022)

    Google Scholar 

  44. Wang, S., Zuccon, G.: An analysis of untargeted poisoning attack and defense methods for federated online learning to rank systems. In: Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval, pp. 215–224 (2023)

    Google Scholar 

  45. Wu, C., Zhu, S., Mitra, P.: Federated unlearning with knowledge distillation. arXiv preprint arXiv:2201.09441 (2022)

  46. Yuan, W., Yin, H., Wu, F., Zhang, S., He, T., Wang, H.: Federated unlearning for on-device recommendation. In: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pp. 393–401 (2023)

    Google Scholar 

  47. Yue, Y., Joachims, T.: Interactively optimizing information retrieval systems as a dueling bandits problem. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1201–1208 (2009)

    Google Scholar 

  48. Zhao, T., King, I.: Constructing reliable gradient exploration for online learning to rank. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 1643–1652 (2016)

    Google Scholar 

  49. Zhuang, S., Qiao, Z., Zuccon, G.: Reinforcement online learning to rank with unbiased reward shaping. Inf. Retr. J. 25(4), 386–413 (2022)

    Article  Google Scholar 

  50. Zhuang, S., Zuccon, G.: Counterfactual online learning to rank. In: Jose, J.M., et al. (eds.) ECIR 2020, Part I. LNCS, vol. 12035, pp. 415–430. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45439-5_28

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Queensland, Brisbane, Australia

    Shuyi Wang & Guido Zuccon

  2. CSIRO, Brisbane, Australia

    Bing Liu

Authors
  1. Shuyi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guido Zuccon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuyi Wang .

Editor information

Editors and Affiliations

  1. Georgetown University, Washington, WA, USA

    Nazli Goharian

  2. University of Pisa, PISA, Pisa, Italy

    Nicola Tonellotto

  3. King's College London, London, UK

    Yulan He

  4. University College London, London, UK

    Aldo Lipani

  5. University of Glasgow, Glasgow, UK

    Graham McDonald

  6. University of Glasgow, Glasgow, UK

    Craig Macdonald

  7. University of Glasgow, Glasgow, UK

    Iadh Ounis

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, S., Liu, B., Zuccon, G. (2024). How to Forget Clients in Federated Online Learning to Rank?. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. Lecture Notes in Computer Science, vol 14610. Springer, Cham. https://doi.org/10.1007/978-3-031-56063-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-56063-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-56062-0

  • Online ISBN: 978-3-031-56063-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation