Skip to main content
SpringerLink
Log in

Reinforcement Learning for Routing

  • Chapter
  • First Online:
Machine Learning Applications in Electronic Design Automation

  • 1796 Accesses

Abstract

Routing has been one of the most critical and challenging steps in electronics design automation (EDA), and existing solutions have historically relied heavily on heuristics and analytical methods. In recent years, reinforcement learning (RL) has emerged as an alternative for use in various routing problems in the space of chip design. RL-based methods tend to outperform existing heuristics and analytical routing algorithms across various metrics including efficiency and solution quality, and a few are able to solve problems that previously remained unsolved. This chapter provides a review of recent RL routing approaches in EDA and shares insights into open challenges and opportunities. Methods covered in this chapter include RL for global routing, RL for detailed routing, RL for standard cell routing, and RL for other related routing problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen, H.-Y., Chang, Y.-W.: Global and detailed routing. In: Electronic Design Automation. Morgan Kaufmann, Los Altos, pp. 687–749 (2009)

    Google Scholar 

  2. Clow, G.W.: A global routing algorithm for general cells. In: 21st Design Automation Conference Proceedings. IEEE, Piscataway (1984)

    Google Scholar 

  3. Chang, Y.-J., Lee, Y.-T., Wang, T.-C.: NTHU-Route 2.0: a fast and stable global router. In: 2008 IEEE/ACM International Conference on Computer-Aided Design. IEEE, Piscataway (2008)

    Google Scholar 

  4. Abel, L.C.: On the ordering of connections for automatic wire routing. IEEE Trans. Comput. 100(11), 1227–1233 (1972)

    Article  Google Scholar 

  5. McMurchie, L., Ebeling, C.: PathFinder: a negotiation-based performance-driven router for FPGAs. In: Reconfigurable Computing, pp. 365–381. Morgan Kaufmann, Los Altos (2008)

    Google Scholar 

  6. Cho, M., Pan, D.Z.: BoxRouter: a new global router based on box expansion and progressive ILP. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 26(12), 2130–2143 (2007)

    Article  Google Scholar 

  7. Pan, M., Chu, C.: FastRoute 2.0: a high-quality and efficient global router. In: 2007 Asia and South Pacific Design Automation Conference. IEEE, Piscataway (2007)

    Google Scholar 

  8. Kastner, R., Bozorgzadeh, E., Sarrafzadeh, M.: Pattern routing: use and theory for increasing predictability and avoiding coupling. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 21(7), 777–790 (2002)

    Article  Google Scholar 

  9. Chen, H.Y., Chang, Y.W.: Global and detailed routing. In: Electronic Design Automation, pp. 687–749. Morgan Kaufmann, Los Altos (2009)

    Google Scholar 

  10. Tang, H., et al.: A survey on Steiner tree construction and global routing for VLSIdesign. IEEE Access 8, 68593–68622 (2020)

    Article  Google Scholar 

  11. Huang, X., et al.: MLXR: multi-layer obstacle-avoiding X-architecture Steiner tree construction for VLSI routing. Sci. China Inform. Sci. 60(1), 1–3 (2017)

    Article  Google Scholar 

  12. Hwang, F.K., Richards, D.S.: Steiner tree problems. Networks 22(1), 55–89 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  13. Ajwani, G. Chu, C., Mak, W.-K.: FOARS: FLUTE based obstacle-avoiding rectilinear Steiner tree construction. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 30(2), 194–204 (2011)

    Article  Google Scholar 

  14. Szymanski, T.G.: Dogleg channel routing is NP-complete. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 4(1), 31–41 (1985)

    Article  Google Scholar 

  15. Lienig, J., Thulasiraman, K.: A genetic algorithm for channel routing in VLSI circuits. Evol. Comput. 1(4), 293–311 (1993)

    Article  Google Scholar 

  16. Chen, Y.K., Liu, M.L.: Three-layer channel routing. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 3(2), 156–163 (1984)

    Article  Google Scholar 

  17. Gao, T., & Liu, C. L. (1996). Minimum crosstalk channel routing. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 15(5), 465–474

    Article  Google Scholar 

  18. Ho, T-T., Sitharama Iyengar, S., Zheng, S.-Q.: A general greedy channel routing algorithm. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 10(2), 204–211 (1991)

    Google Scholar 

  19. Liao, H., Zhang, W., Dong, X., Poczos, B., Shimada, K., Burak Kara, L.: A deep reinforcement learning approach for global routing. J. Mech. Design 142(6), 061701 (2020)

    Article  Google Scholar 

  20. Liu, J., Chen, G., Young, E.F.: REST: constructing rectilinear Steiner minimum tree via reinforcement learning. In: 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, Piscataway (2021)

    Google Scholar 

  21. Liao, H., Dong, Q., Dong, X., Zhang, W., Zhang, W., Qi, W., Fallon, E., Kara, L.B. (2020). Attention routing: track-assignment detailed routing using attention-based reinforcement learning. In: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, vol. 84003, p. V11AT11A002. American Society of Mechanical Engineers

    Google Scholar 

  22. Liao, H., Dong, Q., Qi, W., Fallon, E., Kara, L. B.: Track-assignment detailed routing using attention-based policy model with supervision. In: Proceedings of the 2020 ACM/IEEE Workshop on Machine Learning for CAD, pp. 105–110 (2020)

    Google Scholar 

  23. Qu, T., Lin, Y., Lu, Z., Su, Y., Wei, Y.: Asynchronous reinforcement learning framework for net order exploration in detailed routing. In: 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1815–1820. IEEE, Piscataway (2021)

    Google Scholar 

  24. Ren, H., Fojtik, M., Durham, N.C., Khailany, B.: NVCell: Generate Standard Cell Layout in Advanced Technology Nodes with Reinforcement Learning

    Google Scholar 

  25. Mnih, V., et al.: Playing atari with deep reinforcement learning (2013). arXiv preprint arXiv:1312.5602

    Google Scholar 

  26. Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks (2015). arXiv preprint arXiv:1506.03134

    Google Scholar 

  27. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks (2017). arXiv preprint arXiv:1710.10903

    Google Scholar 

  28. Chen, G., Pui, C.W., Li, H., Young, E.F.: Dr. CU: detailed routing by sparse grid graph and minimum-area-captured path search. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 39(9), 1902–1915 (2019)

    Google Scholar 

  29. Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937. PMLR (2016)

    Google Scholar 

  30. Ren, H., Fojtik, M.: Standard cell routing with reinforcement learning and genetic algorithm in advanced technology nodes. In: Proceedings of the 26th Asia and South Pacific Design Automation Conference, pp. 684–689 (2021)

    Google Scholar 

  31. Sherwani, N.A.: Algorithms for VLSI physical design automation. Springer, Berlin (2012)

    MATH  Google Scholar 

  32. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). arXiv preprint arXiv:1707.06347

    Google Scholar 

  33. Mammeri, Z.: Reinforcement learning based routing in networks: review and classification of approaches. IEEE Access 7, 55916–55950 (2019)

    Article  Google Scholar 

  34. Tamar, A., Wu, Y., Thomas, G., Levine, S., Abbeel, P.: Value iteration networks (2016). arXiv preprint arXiv:1602.02867.

    Google Scholar 

  35. Sykora, Q., Ren, M., Urtasun, R.: Multi-agent routing value iteration network. In: International Conference on Machine Learning, pp. 9300–9310. PMLR (2020)

    Google Scholar 

  36. Kool, W., Van Hoof, H., Welling, M.: Attention, learn to solve routing problems! (2018). arXiv preprint arXiv:1803.08475

    Google Scholar 

  37. Frobeen, L.: Asynchronous Methods for Deep Reinforcement Learning (2017)

    Google Scholar 

  38. Mazyavkina, N., Sviridov, S., Ivanov, S., Burnaev, E.: Reinforcement learning for combinatorial optimization: a survey. Comput. Oper. Res. 134, 105400 (2021)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, USA

    Haiguang Liao & Levent Burak Kara

Authors
  1. Haiguang Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Levent Burak Kara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haiguang Liao .

Editor information

Editors and Affiliations

  1. NVIDIA, Austin, TX, USA

    Haoxing Ren

  2. Texas A&M University, College Station, TX, USA

    Jiang Hu

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Liao, H., Kara, L.B. (2022). Reinforcement Learning for Routing. In: Ren, H., Hu, J. (eds) Machine Learning Applications in Electronic Design Automation. Springer, Cham. https://doi.org/10.1007/978-3-031-13074-8_11

Download citation

Publish with us

Policies and ethics

Search

Navigation