Skip to main content
SpringerLink
Log in

Adaptive load balancing in learning-based approaches for many-core embedded systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Adaptive routing algorithms improve network performance by distributing traffic over the whole network. However, they require congestion information to facilitate load balancing. To provide local and global congestion information, we propose a learning method based on dual reinforcement learning approach. This information can be dynamically updated according to the changing traffic condition in the network by propagating data and learning packets. We utilize a congestion detection method which updates the learning rate according to the congestion level. This method calculates the average number of free buffer slots in each switch at specific time intervals and compares it with maximum and minimum values. Based on the comparison result, the learning rate sets to a value between 0 and 1. If a switch gets congested, the learning rate is set to a high value, meaning that the global information is more important than local. In contrast, local is more emphasized than global information in non-congested switches. Results show that the proposed approach achieves a significant performance improvement over the traditional Q-routing, DRQ-routing, DBAR and Dynamic XY algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ni LM, McKinley PK (1993) A survey of wormhole routing techniques in direct networks. Computer 26(2):62–76

    Article  Google Scholar 

  2. Ebrahimi M, Daneshtalab M, Liljeberg P, Plosila J, Tenhunen H (2011) Agent-based on-chip network using efficient selection method. In: Proceedings of 19th IFIP/IEEE International Conference on very large scale integration (VLSI-SoC), pp 284–289.

  3. Dehyadegari M et al. (2011) An adaptive fuzzy logic-based routing algorithm for networks-on-chip. In: Proceedings of 13th IEEE/NASA-ESA International Conference on adaptive hardware and systems (AHS), pp 208–214.

  4. Sutton RS, Barto AG (2000) Reinforcement learning. MIT Press, Cambridge, An introduction

    Google Scholar 

  5. Watkins CJCH, Dayan P (1992) Q-Learning. In: Proceedings on machine learning, pp 279–292.

  6. Boyan JA, Littman ML (1994) Packet routing in dynamically changing networks: a reinforcement learning approach. Adv Neural Inf Process Syst 6:671–678

    Google Scholar 

  7. Kumar S, Miikkulainen R (1997) Dual reinforcement Q-routing: an on-line adaptive routing algorithm. In: Proceedings of the artificial neural networks in engineering Conference, pp 231–238.

  8. Schonwald T, Zimmermann J, Bringmann O (2007) Fully adaptive fault-tolerant routing algorithm for network-on-chip architectures. In Euromicro Conference on digital system design architectures, methods and tools (DSD), Lübeck, pp 527–534.

  9. Chiu G-M (2000) The odd-even turn model for adaptive routing. IEEE Trans Parallel Distrib Syst 11(7):729–738

    Article  Google Scholar 

  10. Ebrahimi M et al. (2012) MAFA: adaptive fault-tolerant routing algorithm for networks-on-chip. In: Proceedings of 15th IEEE Euromicro Conference on Digital System Design (DSD), pp 201–206.

  11. Boura YM, Das CR (1994) Efficient fully adaptive wormhole routing in n-dimensional meshes. In: Proceedings of the 14th international conference on distributed computing systems (ICDCS). Pozman, pp 589–596.

  12. Feng W, Shin KG (1997) Impact of selection functions on routing algorithm performance in multicomputer networks. In: International Conference on Supercomputing, pp 132–139.

  13. Badr HG, Podar S (1989) An optimal shortest-path routing policy for network computers with regular mesh-connected topologies. IEEE Trans Parallel Distrib Syst 38(10):1362–1371

    MathSciNet  Google Scholar 

  14. Li M, Zeng Q, Jone W (2006) DyXY–a proximity congestion-aware deadlock-free dynamic routing method for network on chip. In: Processing of design automation conference (DAC). San Francisco, pp 849–852.

  15. Hu J, Marculescu R (2004) DyAD–Smart routing for network-on-chip. In: Processing of design automation conference (DAC). San Diego, pp 260–263.

  16. Dally WJ, Aoki H (1993) Deadlock-free adaptive routing in multicomputer networks using virtual channels. IEEE Trans Parallel Distrib Syst 4(4):466–475

    Article  Google Scholar 

  17. Singh A, Dally WJ, Gupta AK, Towles B (2003) GOAL: A load-balanced adaptive routing algorithm for torus networks. In: International Symposium on Computer, Architecture, pp 194–205.

  18. Kim J, Park D, Theocharides T, Vijaykrishnan N, Das CR (2005) A low latency router supporting adaptivity for on-chip interconnects. Proceedings of the 42nd annual design automation conference (DAC). ACM, New York, pp 559–564

    Chapter  Google Scholar 

  19. Ascia G (2008), Implementation and analysis of a new selection strategy for adaptive routing in networks-on-chip. IEEE Trans Comput 57(I.6):809–820.

    Google Scholar 

  20. Gratz P, Grot B, Keckler SW (2008) Regional congestion awareness for load balance in networks-on-chip. In: Proceeding of the 14th international symposium on high-performance computer architecture. Salt Lake, City, pp 203–214.

  21. Ma S et al. (2011) DBAR: an efficient routing algorithm to support multiple concurrent applications in networks-on-chip. In: Proceeding of 38th annual international symposium on computer architecture (ISCA). San Jose, pp 413–424.

  22. Ebrahimi M et al. (2012) CATRA—congestion aware trapezoid-based routing algorithm for on-chip networks. In: Proceeding of design, automation & test in Europe conference & exhibition (DATE). Dresden, pp 320–325.

  23. Choi SP, Yeung D-Y (1996) Predictive Q-routing: a memory-based reinforcement learning approach to adaptive traffic control. Adv Neural Inf Process Syst 8(NIPS8):945–951

    Google Scholar 

  24. Kumar S, Miikkulainen R (1998) Confidence-based Q-routing: an on-line adaptive network routing algorithm. Smart engineering systems: neural networks, fuzzy logic, data mining, and evolutionary programming 8:147–152

    Google Scholar 

  25. Kumar S, Miikkulainen R (1997), Dual reinforcement Q-routing: an on-line adaptive routing algorithm.In: Proceedings of the Artificial Neural Networks in, Engineering Conference, pp 231–238.

  26. Feng C, Lu Z, Jantsch A, Li J, Zhang M (2010) A reconfigurable fault-tolerant deflection routing algorithm based on reinforcement learning for network-on-chip. In: Proceedings of NoCArc, pp 11–16.

  27. Majer M et al. (2005) Packet routing in dynamically changing networks on chip. In: Proceedings of the 19th International Parallel and Distributed Processing Symposium (IPDPS). Denver, USA.

  28. Paliwal KK, George JS, Rameshan N, Laxmi V, Gaur MS, Janyani V, Narasimhan R (2009) Implementation of QoS aware Q-routing algorithm for network-on-chip. In: Contemporary computing. Springer, Berlin, Heidelberg, pp 370–380

  29. Puthal MK, Singh V, Gaur MS, Laxmi V (2011) C-routing: an adaptive hierarchical NoC routing methodology. In: IEEE/IFIP 19th international conference on VLSI and system-on-chip (VLSI-SoC). Hong Kong, pp 392–397.

  30. Farahnakian F, Ebrahimi M, Daneshtalab M, Liljeberg P, Plosila J (2011) Q-learning based congestion-aware routing algorithm for on-chip network. In: Proceedings of 2nd IEEE international conference on networked embedded systems for enterprise applications (NESEA). Fremantle, pp 1–7.

  31. Farahnakian F, Ebrahimi M, Daneshtalab M, Plosila J, Liljeberg P (2012) Adaptive reinforcement learning method for networks-on-chip. In: Proceedings of 12th IEEE international conference on embedded computer systems: architectures, modeling, and simulation (SAMOS XII). Samos, pp 236–243.

  32. Ebrahimi M, Daneshtalab M, Farahnakian F, Liljeberg P, Plosila J, Palesi M, Tenhunen H (2012) HARAQ: congestion-aware learning model for highly adaptive routing algorithm in on-chip networks. In: Proceedings of 6th ACM/IEEE International Symposium on Networks-on-Chip (NOCS), pp 19–26.

  33. Farahnakian F, Ebrahimi M, Daneshtalab M, Liljeberg P, Plosila J (2014) Bi-LCQ: a low-weight clustering-based Q-learning approach for NoCs. Elsevier J Microprocess Microsyst (MICPRO) 38:64–75

    Article  Google Scholar 

  34. Farahnakian F, Ebrahimi M, Daneshtalab M, Liljeberg P, Plosila J (2012) Optimized Q-learning model for distributing traffic in on-chip networks. In: International Conference on Networked Embedded Systems for Enterprise Applications (NESEA), UK, pp 1–8.

  35. Varga A et al. (2001) The OMNeT++ discrete event simulation system. In: Proceedings of the European Simulation Multiconference (ESM’2001), pp 319–324.

  36. Woo SC et al. (1995) The splash-2 programs: characterization and methodological considerations. In: Proceedings of Computer Architecture (ISCA), pp 24–36.

  37. Martin MK, Sorin DJ, Beckmann BM et al (2005) Multifacet’s general execution driven multiprocessor simulator (GEMS) toolset. SIGARCH Comput Archit News 33(4):92–99

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Turku, Turku, Finland

    F. Farahnakian, M. Ebrahimi, M. Daneshtalab, P. Liljeberg & J. Plosila

Authors
  1. F. Farahnakian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Ebrahimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Daneshtalab
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. P. Liljeberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Plosila
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Farahnakian.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Farahnakian, F., Ebrahimi, M., Daneshtalab, M. et al. Adaptive load balancing in learning-based approaches for many-core embedded systems. J Supercomput 68, 1214–1234 (2014). https://doi.org/10.1007/s11227-014-1166-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-014-1166-1

Keywords

Search

Navigation