Skip to main content
SpringerLink
Log in

Federated Learning for Resource-Constrained IoT Devices: Panoramas and State of the Art

  • Chapter
  • First Online:
Federated and Transfer Learning

Part of the book series: Adaptation, Learning, and Optimization ((ALO,volume 27))

Abstract

Nowadays, devices are equipped with advanced sensors with higher processing and computing capabilities. Besides, widespread Internet availability enables communication among sensing devices that results the generation of vast amounts of data on edge devices to drive Internet-of-Things (IoT), crowdsourcing, and other emerging technologies. The extensive amount of collected data can be preprocessed, scaled, classified, and finally, used for predicting future events with machine learning (ML) methods. In traditional ML approaches, data is sent to and processed in a central server, which encounters communication overhead, processing delay, privacy leakage, and security issues. To overcome these challenges, each client can be trained locally based on its available data and by learning from the global model. This decentralized learning approach is referred to as federated learning (FL). However, in large-scale networks, there may be clients with varying computational resource capabilities. This may lead to implementation and scalability challenges for FL techniques. In this paper, we first introduce some recently implemented real-life applications of FL underlying the applications that are suitable for FL-based resource-constrained IoT environments. We then emphasize the core challenges of implementing the FL algorithms from the perspective of resource limitations (e.g., memory, bandwidth, and energy budget) of client devices. We finally discuss open issues associated with FL for resource-constrained environments and highlight future directions in the FL domain concerning resource-constrained devices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hard A, Rao K, Mathews R, Ramaswamy et al (2018) Federated learning for mobile keyboard prediction. arXiv:1811.03604

  2. Leroy D, Coucke A et al (2019) Federated learning for keyword spotting. In: IEEE ICASSP

    Google Scholar 

  3. Lim WYB, Luong NC et al (2019) Federated learning in mobile edge networks: a comprehensive survey. arXiv:1909.11875

  4. Park J, Wang S et al (2019) Distilling on-device intelligence at the network edge. arXiv:1908.05895

  5. Das A, Brunschwiler T (2019) Privacy is what we care about: experimental investigation of federated learning on edge devices. In: AIChallengeIoT

    Google Scholar 

  6. Xu Z, Li L et al (2019) Exploring federated learning on battery-powered devices. In: ACM TURC

    Google Scholar 

  7. Imteaj A, Amini MH (2021) Fedparl: client activity and resource-oriented lightweight federated learning model for resource-constrained heterogeneous iot environment. Front Commun Netw 2:10

    Google Scholar 

  8. Xu Z, Yang Z et al (2019) Elfish: resource-aware federated learning on heterogeneous edge devices. arXiv:1912.01684

  9. Wang S, Tuor T et al (2019) Adaptive federated learning in resource constrained edge computing systems. IEEE JSAC 37(6):1205–1221

    Google Scholar 

  10. What does it take to train deep learning models on-device? (2018)

    Google Scholar 

  11. Bonawitz K, Eichner et al (2019) Towards federated learning at scale: system design. arXiv:1902.01046

  12. McMahan HB, Moore E et al (2016) Communication-efficient learning of deep networks from decentralized data. arXiv:1602.05629

  13. Huang L, Yin Y et al (2018) Loadaboost: loss-based adaboost federated machine learning on medical data. arXiv:1811.12629

  14. Yang T, Andrew G et al (2018) Applied federated learning: improving google keyboard query suggestions. arXiv:1812.02903

  15. Chen F, Dong Z et al (2018) Federated meta-learning for recommendation. arXiv:1802.07876

  16. Imteaj A, Khan I, Khazaei J, Amini MH (2021) Fedresilience: a federated learning application to improve resilience of resource-constrained critical infrastructures. Electronics, 10(16)

    Google Scholar 

  17. Yang Q, Liu Y et al (2019) Federated machine learning: concept and applications. ACM Trans TIST 10(2):12

    Google Scholar 

  18. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  19. Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, Cui C, Corrado G, Thrun S, Dean J (2019) A guide to deep learning in healthcare. Nat Med 25(1):24–29

    Article  Google Scholar 

  20. Shen D, Guorong W, Suk H-I (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248

    Article  Google Scholar 

  21. Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75

    Article  Google Scholar 

  22. General Data Protection Regulation (2018) General data protection regulation (gdpr). Intersoft Consulting Accessed in October, 24(1)

    Google Scholar 

  23. Zhan Y, Li P, Guo S (2020) Experience-driven computational resource allocation of federated learning by deep reinforcement learning. In: 2020 IEEE international parallel and distributed processing symposium (IPDPS). IEEE, pp 234–243

    Google Scholar 

  24. He C, Annavaram M, Avestimehr S (2020) Group knowledge transfer: federated learning of large cnns at the edge. arXiv:2007.14513

  25. McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. PMLR, pp 1273–1282

    Google Scholar 

  26. Gupta O, Raskar R (2018) Distributed learning of deep neural network over multiple agents. J Netw Comput Appl 116:1–8

    Article  Google Scholar 

  27. Ahmed KM, Imteaj A, Amini MH (2021) Federated deep learning for heterogeneous edge computing. In: 2021 20th IEEE international conference on machine learning and applications (ICMLA). IEEE

    Google Scholar 

  28. Hu R, Guo Y, Ratazzi EP, Gong Y (2020) Differentially private federated learning for resource-constrained internet of things. arXiv:2003.12705

  29. Khan S, Yairi T (2018) A review on the application of deep learning in system health management. Mech Syst Signal Process 107:241–265

    Article  Google Scholar 

  30. Mahdavifar S, Ghorbani AA (2019) Application of deep learning to cybersecurity: a survey. Neurocomputing 347:149–176

    Article  Google Scholar 

  31. Ahmed KM, Eslami T, Saeed F, Amini MH (2021) Deepcovidnet: deep convolutional neural network for covid-19 detection from chest radiographic images. In: 2021 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE, pp 1703–1710

    Google Scholar 

  32. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489

    Article  Google Scholar 

  33. Russakovsky O, Deng J, Hao S, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  34. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  35. Jiang J, Zhai CX (2007) Instance weighting for domain adaptation in nlp. ACL

    Google Scholar 

  36. Gao J, Fan W, Jiang J, Han J (2008) Knowledge transfer via multiple model local structure mapping. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 283–291

    Google Scholar 

  37. Argyriou A, Pontil M, Ying Y, Micchelli C (2007) A spectral regularization framework for multi-task structure learning. Adv Neural Inf Proc Syst 20

    Google Scholar 

  38. Mihalkova L, Huynh T, Mooney RJ (2007) Mapping and revising markov logic networks for transfer learning. Aaai 7:608–614

    Google Scholar 

  39. Li H, Ota K et al (2018) Learning iot in edge: deep learning for the internet of things with edge computing. IEEE Netw 32(1):96–101

    Article  Google Scholar 

  40. Cui L, Yang S et al (2018) A survey on application of machine learning for internet of things. J M L Cybern 9(8):1399–1417

    Article  Google Scholar 

  41. Haddadpour F, Kamani MM et al (2019) Trading redundancy for communication: speeding up distributed sgd for non-convex optimization. In: ICML

    Google Scholar 

  42. Huang J, Qian F et al (2013) An in-depth study of lte: effect of network protocol and application behavior on performance. ACM SIGCOMM CCR 43(4):363–374

    Article  MathSciNet  Google Scholar 

  43. Ma C, Konečnỳ J et al (2017) Distributed optimization with arbitrary local solvers. Optim Methods Softw 32(4):813–848

    Article  MathSciNet  Google Scholar 

  44. Imteaj A, Amini MH (2019) Distributed sensing using smart end-user devices: pathway to federated learning for autonomous iot. In: 2019 international conference on computational science and computational intelligence (CSCI). IEEE, pp 1156–1161

    Google Scholar 

  45. Konečnỳ J, McMahan HB et al (2016) Federated learning: strategies for improving communication efficiency. arXiv:1610.05492

  46. Li T, Sahu AK et al (2019) Federated learning: challenges, methods, and future directions. arXiv:1908.07873

  47. Thrun S et al (2012) Learning to learn. Springer Science & Business Media, Berlin

    MATH  Google Scholar 

  48. Caruana R (1997) Multitask learning. Mach Learn 28(1):41–75

    Article  MathSciNet  Google Scholar 

  49. Corinzia L et al (2019) Variational federated multi-task learning. arXiv:1906.06268

  50. Wu S, Li G et al (2018) Training and inference with integers in deep neural networks. arXiv:1802.04680

  51. Jiang Y, Wang S et al (2019) Model pruning enables efficient federated learning on edge devices. arXiv:1909.12326

  52. Yan G, Wang H, Li J (2021) Critical learning periods in federated learning. arXiv:2109.05613

  53. Thakker U, Beu J et al (2019) Compressing rnns for iot devices by 15-38x using kronecker products. arXiv:1906.02876

  54. Thakker U, Whatmough P, Liu Z, Mattina M, Beu J (2021) Doping: a technique for extreme compression of lstm models using sparse structured additive matrices. In: Smola A, Dimakis A, Stoica I (eds), Proceedings of machine learning and systems, vol 3, pp 533–549

    Google Scholar 

  55. Gope D, Beu J, Thakker U, Mattina M (2020) Ternary mobilenets via per-layer hybrid filter banks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) workshops

    Google Scholar 

  56. Xiong G, Yan G, Li J (2021) Straggler-resilient distributed machine learning with dynamic backup workers. arXiv:2102.06280

  57. Imteaj A, Amini MH (2020) Fedar: activity and resource-aware federated learning model for distributed mobile robots. In: 2020 19th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 1153–1160

    Google Scholar 

  58. Imteaj A (2020) Distributed machine learning for collaborative mobile robots: Phd forum abstract. In: Proceedings of the 18th conference on embedded networked sensor systems, SenSys ’20, New York, NY, USA, 2020. Association for Computing Machinery, pp 798–799

    Google Scholar 

  59. Gu Z, Jamjoom H et al (2019) Reaching data confidentiality and model accountability on the caltrain. In: IEEE DSN

    Google Scholar 

  60. Chen M, Yang Z et al (2019) A joint learning and communications framework for federated learning over wireless networks. arXiv:1909.07972

  61. Sprague MR, Jalalirad A et al (2018) Asynchronous federated learning for geospatial applications. In: ECML-PKDD

    Google Scholar 

  62. Eliazar II, Sokolov IM (2010) Measuring statistical heterogeneity: the pietra index. Physica A: Stat Mech App 389(1):117–125

    Article  Google Scholar 

  63. Kumar A, Goyal S et al (2017) Resource-efficient machine learning in 2 kb ram for the internet of things. In: ICML

    Google Scholar 

  64. Dettmers T, Lewis M, Shleifer S, Zettlemoyer L (2021) 8-bit optimizers via block-wise quantization

    Google Scholar 

  65. Anonymous (2022) Logarithmic unbiased quantization: practical 4-bit training in deep learning. In: Submitted to the tenth international conference on learning representations. Under review

    Google Scholar 

  66. Raju R, Gope D, Thakker U, Beu J (2020) Understanding the impact of dynamic channel pruning on conditionally parameterized convolutions. In: Proceedings of the 2nd international workshop on challenges in artificial intelligence and machine learning for internet of things, AIChallengeIoT ’20, New York, NY, USA, 2020. Association for Computing Machinery, pp 27–33

    Google Scholar 

  67. Huang X, Thakker U, Gope D, Beu J (2020) Pushing the envelope of dynamic spatial gating technologies. AIChallengeIoT ’20, New York, NY, USA, 2020. Association for Computing Machinery, pp 21–26

    Google Scholar 

  68. Zhang Y, Duchi J et al (2015) Divide and conquer kernel ridge regression: a distributed algorithm with minimax optimal rates. JMLR 16(1):3299–3340

    MathSciNet  MATH  Google Scholar 

  69. Guha N, Talwlkar A et al (2019) One-shot federated learning. arXiv:1902.11175

  70. Kim H, Park J et al (2019) Blockchained on-device federated learning. IEEE Commun Lett

    Google Scholar 

  71. Xu R, Chen Y, Li J (2020) MicroFL: a lightweight, secure-by-design edge network fabric for decentralized IoT systems. In: NDSS

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Knight Foundation School of Computing and Information Sciences, Sustainability, Optimization, and Learning for InterDependent networks laboratory (SOLID Lab), Florida International University, 11200 SW 8th St, ECS 354, Miami, FL, 33199, USA

    Ahmed Imteaj, Khandaker Mamun Ahmed & M. Hadi Amini

  2. Deep Learning Research, SambaNova Systems, Palo Alto, CA, USA

    Urmish Thakker

  3. IBM T. J. Watson Research Center, Yorktown Heights, NY, USA

    Shiqiang Wang

  4. Binghamton University, State University of New York, Albany, NY, USA

    Jian Li

Authors
  1. Ahmed Imteaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khandaker Mamun Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Urmish Thakker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shiqiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. Hadi Amini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Hadi Amini .

Editor information

Editors and Affiliations

  1. Faculty of Computer Science, University of New Brunswick, Fredericton, NB, Canada

    Roozbeh Razavi-Far

  2. Department of Computer Science, Western University, London, ON, Canada

    Boyu Wang

  3. University of Alberta and the Alberta Machine Intelligence Institute (Amii), Edmonton, AB, Canada

    Matthew E. Taylor

  4. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Kowloon, China

    Qiang Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Imteaj, A., Mamun Ahmed, K., Thakker, U., Wang, S., Li, J., Amini, M.H. (2023). Federated Learning for Resource-Constrained IoT Devices: Panoramas and State of the Art. In: Razavi-Far, R., Wang, B., Taylor, M.E., Yang, Q. (eds) Federated and Transfer Learning. Adaptation, Learning, and Optimization, vol 27. Springer, Cham. https://doi.org/10.1007/978-3-031-11748-0_2

Download citation

Publish with us

Policies and ethics

Search

Navigation