Skip to main content

Collaborative Fairness in Federated Learning

Part of the Lecture Notes in Computer Science book series (LNAI,volume 12500)

Abstract

In current deep learning paradigms, local training or the Standalone framework tends to result in overfitting and thus low utility. This problem can be addressed by Distributed or Federated Learning (FL) that leverages a parameter server to aggregate local model updates. However, all the existing FL frameworks have overlooked an important aspect of participation: collaborative fairness. In particular, all participants can receive the same or similar models, even the ones who contribute relatively less, and in extreme cases, nothing. To address this issue, we propose a novel Collaborative Fair Federated Learning (CFFL) framework which utilizes reputations to enforce participants to converge to different models, thus ensuring fairness and accuracy at the same time. Extensive experiments on benchmark datasets demonstrate that CFFL achieves high fairness and performs comparably to the Distributed framework and better than the Standalone framework.

Keywords

  • Collaborative learning
  • Fairness
  • Reputation

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-63076-8_14
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   59.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-63076-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   79.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

Notes

  1. 1.

    http://yann.lecun.com/exdb/mnist/.

  2. 2.

    http://archive.ics.uci.edu/ml/datasets/Adult.

References

  1. Cummings, R., Gupta, V., Kimpara, D., Morgenstern, J.: On the compatibility of privacy and fairness. In: Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, pp. 309–315 (2019)

    Google Scholar 

  2. Gollapudi, S., Kollias, K., Panigrahi, D., Pliatsika, V.: Profit sharing and efficiency in utility games. In: ESA, pp. 1–16 (2017)

    Google Scholar 

  3. Hardy, S., et al.: Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. CoRR. arXiv:1711.10677 (2017)

  4. Jagielski, M., et al.: Differentially private fair learning. arXiv preprint arXiv:1812.02696 (2018)

  5. Kairouz, P., et al.: Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 (2019)

  6. Kantarcioglu, M., Clifton, C.: Privacy-preserving distributed mining of association rules on horizontally partitioned data. IEEE Trans. Knowl. Data Eng. 16(9), 1026–1037 (2004)

    CrossRef  Google Scholar 

  7. Li, D., Wang, J.: FedMD: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019)

  8. Li, T., Sahu, A.K., Talwalkar, A., Smith, V.: Federated learning: challenges, methods, and future directions. CoRR. arXiv:1908.07873 (2019)

  9. Li, T., Sanjabi, M., Smith, V.: Fair resource allocation in federated learning. In: ICLR (2020)

    Google Scholar 

  10. Lingjuan Lyu, X.X., Wang, Q.: Collaborative fairness in federated learning (2020). https://arxiv.org/abs/2008.12161v1

  11. Lyu, L., Li, Y., Nandakumar, K., Yu, J., Ma, X.: How to democratise and protect AI: fair and differentially private decentralised deep learning. IEEE Trans. Dependable Secure Compu. (2020)

    Google Scholar 

  12. Lyu, L., Yu, H., Yang, Q.: Threats to federated learning: a survey. arXiv preprint arXiv:2003.02133 (2020)

  13. Lyu, L., et al.: Towards fair and privacy-preserving federated deep models. IEEE Trans. Parallel Distrib. Syst. 31(11), 2524–2541 (2020)

    CrossRef  Google Scholar 

  14. McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282 (2017)

    Google Scholar 

  15. Mohri, M., Sivek, G., Suresh, A.T.: Agnostic federated learning. In: International Conference on Machine Learning, pp. 4615–4625 (2019)

    Google Scholar 

  16. Regatti, J., Gupta, A.: Befriending the byzantines through reputation scores. arXiv preprint arXiv:2006.13421 (2020)

  17. Richardson, A., Filos-Ratsikas, A., Faltings, B.: Rewarding high-quality data via influence functions. arXiv preprint arXiv:1908.11598 (2019)

  18. Shapley, L.S.: A value for n-person games. In: Contributions to the Theory of Games, vol. 2, no. 28, pp. 307–317 (1953)

    Google Scholar 

  19. Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321. ACM (2015)

    Google Scholar 

  20. Sim, R.H.L., Zhang, Y., Chan, M.C., Low, B.K.H.: Collaborative machine learning with incentive-aware model rewards. In: ICML (2020)

    Google Scholar 

  21. Vaidya, J., Clifton, C.: Privacy preserving association rule mining in vertically partitioned data. In: KDD, pp. 639–644 (2002)

    Google Scholar 

  22. Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. (TIST) 10(2), 1–19 (2019)

    CrossRef  Google Scholar 

  23. Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., Yu, H.: Federated Learning. Morgan & Claypool Publishers (2019)

    Google Scholar 

  24. Yang, S., Wu, F., Tang, S., Gao, X., Yang, B., Chen, G.: On designing data quality-aware truth estimation and surplus sharing method for mobile crowdsensing. IEEE J. Sel. Areas Commun. 35(4), 832–847 (2017)

    CrossRef  Google Scholar 

  25. Yu, H., et al.: A fairness-aware incentive scheme for federated learning. In: Proceedings of the 3rd AAAI/ACM Conference on AI, Ethics, and Society (AIES 2020), pp. 393–399 (2020)

    Google Scholar 

  26. Zhang, J., Li, C., Robles-Kelly, A., Kankanhalli, M.: Hierarchically fair federated learning. arXiv preprint arXiv:2004.10386 (2020)

  27. Zhao, L., Wang, Q., Zou, Q., Zhang, Y., Chen, Y.: Privacy-preserving collaborative deep learning with unreliable participants. IEEE Trans. Inf. Forensics Secur. 15, 1486–1500 (2019)

    CrossRef  Google Scholar 

  28. Zhao, Y., et al.: Privacy-preserving blockchain-based federated learning for IoT devices. IEEE Internet Things J. (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Lingjuan Lyu or Han Yu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Lyu, L., Xu, X., Wang, Q., Yu, H. (2020). Collaborative Fairness in Federated Learning. In: Yang, Q., Fan, L., Yu, H. (eds) Federated Learning. Lecture Notes in Computer Science(), vol 12500. Springer, Cham. https://doi.org/10.1007/978-3-030-63076-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63076-8_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63075-1

  • Online ISBN: 978-3-030-63076-8

  • eBook Packages: Computer ScienceComputer Science (R0)