Skip to main content
SpringerLink
Log in

RL for Placement and Partitioning

  • Chapter
  • First Online:
Machine Learning Applications in Electronic Design Automation

  • 1774 Accesses

Abstract

This chapter starts by describing the problem of chip placement, a time-consuming stage in the overall chip design process and a challenging combinatorial optimization problem. Next, this chapter delves briefly into the six decades of prior work on this important topic. The heart of the chapter is an overview of deep RL, a primer on how to formulate chip placement as a deep RL problem, and a detailed description of a recent RL-based approach to chip placement. The chapter concludes with a discussion of other applications for RL-based methods and their implications for the future of chip design.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Addanki, R., Venkatakrishnan, S.B., Gupta, S., Mao, H., Alizadeh, M.: Learning generalizable device placement algorithms for distributed machine learning. In: Advances in Neural Information Processing Systems, vol. 32, pp. 3981–3991 (2019)

    Google Scholar 

  2. Agnihotri, A., Ono, S., Madden, P.: Recursive bisection placement: Feng Shui 5.0 implementation details. In: Proceedings of the International Symposium on Physical Design, pp. 230–232 (2005). https://doi.org/10.1145/1055137.1055186

  3. Alpert, C., Caldwell, A., Chan, T., Huang, D.H., Kahng, A., Markov, I., Moroz, M.: Analytical engines are unnecessary in top-down partitioning-based placement. VLSI Design 10 (2002). https://doi.org/10.1155/1999/93607

  4. Barrett, T.D., Clements, W.R., Foerster, J.N., Lvovsky, A.I.: Exploratory combinatorial optimization with reinforcement learning (2020). arXiv preprint arXiv:1909.04063v2

    Google Scholar 

  5. Bello, I., Pham, H., Le, Q.V., Norouzi, M., Bengio, S.: Neural Combinatorial Optimization with Reinforcement Learning (2016)

    Google Scholar 

  6. Brenner, U., Struzyna, M., Vygen, J.: BonnPlace: placement of leading-edge chips by advanced combinatorial algorithms. Trans. Comp.-Aided Des. Integr. Circuits Syst. 27(9), 1607–1620 (2008). https://doi.org/10.1109/TCAD.2008.927674

    Article  Google Scholar 

  7. Breuer, M.A.: A class of min-cut placement algorithms. In: Proceedings of the 14th Design Automation Conference, DAC 1977, pp. 284–290. IEEE Press (1977)

    Google Scholar 

  8. Caldwell, A.E., Kahng, A.B., Mantik, S., Markov, I.L., Zelikovsky, A.: On wirelength estimations for row-based placement. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 18(9), 1265–1278 (1999). https://doi.org/10.1109/43.784119

    Article  Google Scholar 

  9. Chen, T., Jiang, Z., Hsu, T., Chen, H., Chang, Y.: NTUplace3: an analytical placer for large-scale mixed-size designs with preplaced blocks and density constraints. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 27(7), 1228–1240 (2008). https://doi.org/10.1109/TCAD.2008.923063

    Article  Google Scholar 

  10. Chen, T.C., Jiang, Z.W., Hsu, T.C., Chen, H.C., Chang, Y.W.: A high-quality mixed-size analytical placer considering preplaced blocks and density constraints. In: Proceedings of the 2006 IEEE/ACM International Conference on Computer-Aided Design, pp. 187–192. Association for Computing Machinery (2006)

    Google Scholar 

  11. Cheng, C., Kahng, A.B., Kang, I., Wang, L.: Replace: Advancing Solution Quality and Routability Validation in Global Placement, pp. 1717–1730 (2019)

    Google Scholar 

  12. Chung-Kuan Cheng, Kuh, E.S.: Module placement based on resistive network optimization. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 3(3), 218–225 (1984). https://doi.org/10.1109/TCAD.1984.1270078

  13. Dunlop, A.E., Kernighan, B.W.: A procedure for placement of standard-cell VLSI circuits. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 4(1), 92–98 (1985). https://doi.org/10.1109/TCAD.1985.1270101

    Article  Google Scholar 

  14. Fiduccia, C.M., Mattheyses, R.M.: A linear-time heuristic for improving network partitions. In: 19th Design Automation Conference, pp. 175–181 (1982). https://doi.org/10.1109/DAC.1982.1585498

  15. Guadarrama, S., Yue, S., Boyd, T., Jiang, J.W., Songhori, E., Tam, T., Mirhoseini, A.: Circuit training: an open-source framework for generating chip floor plans with distributed deep reinforcement learning (2021). https://github.com/google_research/circuit_training. Accessed 21 Dec 2021

  16. Hu, B., Marek-Sadowska, M.: Multilevel fixed-point-addition-based VLSI placement. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 24(8), 1188–1203 (2005). https://doi.org/10.1109/TCAD.2005.850802

    Article  Google Scholar 

  17. Kahng, A.B., Qinke Wang: Implementation and extensibility of an analytic placer. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 24(5), 734–747 (2005). https://doi.org/10.1109/TCAD.2005.846366

    Article  MathSciNet  Google Scholar 

  18. Kahng, A.B., Reda, S.: A tale of two nets: Studies of wirelength progression in physical design. In: Proceedings of the 2006 International Workshop on System-Level Interconnect Prediction, SLIP ’06, pp. 17–24. Association for Computing Machinery, New York (2006). https://doi.org/10.1145/1117278.1117282

  19. Kahng, A.B., Reda, S., Qinke Wang: Architecture and details of a high quality, large-scale analytical placer. In: ICCAD-2005. IEEE/ACM International Conference on Computer-Aided Design, pp. 891–898 (2005). https://doi.org/10.1109/ICCAD.2005.1560188

  20. Kahng, A.B., Wang, Q.: An analytic placer for mixed-size placement and timing-driven placement. In: IEEE/ACM International Conference on Computer Aided Design, ICCAD-2004, pp. 565–572 (2004). https://doi.org/10.1109/ICCAD.2004.1382641

    Google Scholar 

  21. Kahng, A.B., Xu, X.: Accurate pseudo-constructive wirelength and congestion estimation. In: Proceedings of the 2003 International Workshop on System-Level Interconnect Prediction, SLIP ’03, pp. 61–68. Association for Computing Machinery, New York (2003). https://doi.org/10.1145/639929.639942

  22. Khalil, E., Dai, H., Zhang, Y., Dilkina, B., Song, L.: Learning combinatorial optimization algorithms over graphs. In: Advances in Neural Information Processing Systems, vol. 30, pp. 6348–6358 (2017)

    Google Scholar 

  23. Kim, M., Lee, D., Markov, I.L.: SimPL: an effective placement algorithm. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 31(1), 50–60 (2012). https://doi.org/10.1109/TCAD.2011.2170567

    Article  Google Scholar 

  24. Kim, M.C., Markov, I.L.: ComPLx: a competitive primal-dual lagrange optimization for global placement. In: Design Automation Conference 2012, pp. 747–755 (2012)

    Google Scholar 

  25. Kim, M.C., Viswanathan, N., Alpert, C.J., Markov, I.L., Ramji, S.: MAPLE: multilevel adaptive placement for mixed-size designs. In: Proceedings of the 2012 ACM International Symposium on International Symposium on Physical Design, ISPD 2012, pp. 193–200. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2160916.2160958

  26. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lin, T., Chu, C., Shinnerl, J.R., Bustany, I., Nedelchev, I.: POLAR: placement based on novel rough legalization and refinement. In: Proceedings of the International Conference on Computer-Aided Design, pp. 357–362. IEEE Press (2013)

    Google Scholar 

  28. Lin, Y., Dhar, S., Li, W., Ren, H., Khailany, B., Pan, D.Z.: Dreamplace: deep learning toolkit-enabled GPU acceleration for modern vlsi placement. In: Proceedings of the 56th Annual Design Automation Conference 2019, DAC ’19. Association for Computing Machinery, New York (2019)

    Google Scholar 

  29. Lu, J., Chen, P., Chang, C.C., Sha, L., Huang, D.J.H., Teng, C.C., Cheng, C.K.: ePlace: Electrostatics-Based Placement Using Fast Fourier Transform and Nesterov’s Method (2015)

    Google Scholar 

  30. Luo, T., Pan, D.Z.: DPlace2.0: A stable and efficient analytical placement based on diffusion. In: 2008 Asia and South Pacific Design Automation Conference, pp. 346–351 (2008). https://doi.org/10.1109/ASPDAC.2008.4483972

  31. Mirhoseini, A., Goldie, A., Pham, H., Steiner B., Le, Q. V. & Dean, J.: A hierarchical model for device placement. In: Proceedings of the International Conference on Learning Representations (2018)

    Google Scholar 

  32. Mirhoseini, A., Pham, H., Le, Q. V., Steiner, B., Larsen, R., Zhou, Y., Kumar, N., Norouzi, M., Bengio, S., Dean, J.: Device placement optimization with reinforcement learning. In: Proceedings of the International Conference on Machine Learning (2017)

    Google Scholar 

  33. Mirhoseini and Goldie, Yazgan, M., Jiang, J.W., Songhori, E., Wang, S., Lee, Y.J., Johnson, E., Pathak, O., Nazi, A., Pak, J., Tong, A., Srinivasa, K., Hang, W., Tuncer, E., V. Le, Q., Laudon, J., Ho, R., Carpenter, R., Dean, J.: A graph placement methodology for fast chip design. Nature 594(7862), 207–212 (2021)

    Google Scholar 

  34. Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T.P., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning (2016)

    Google Scholar 

  35. Obermeier, B., Ranke, H., Johannes, F.M.: Kraftwerk: a versatile placement approach. In: Proceedings of the 2005 International Symposium on Physical Design, pp. 242–244. Association for Computing Machinery, New York (2005). https://doi.org/10.1145/1055137.1055190

  36. Paliwal, A., Gimeno, F., Nair, V., Li, Y., Lubin, M., Kohli, P., Vinyals, O.: Reinforced genetic algorithm learning for optimizing computation graphs. In: Proceedings of International Conference on Learning Representations (2020)

    Google Scholar 

  37. Roy, J.A., Papa, D.A., Markov, I.L.: Capo: Congestion-Driven Placement for Standard-cell and RTL Netlists with Incremental Capability, pp. 97–133. Springer, Boston (2007)

    Google Scholar 

  38. Sarrafzadeh, M., Wang, M., Yang, X.: Dragon: A Placement Framework, pp. 57–89. Springer, Boston (2003)

    Google Scholar 

  39. Schulman, J., Levine, S., Moritz, P., Jordan, M.I., Abbeel, P.: Trust region policy optimization (2015)

    Google Scholar 

  40. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017)

    Google Scholar 

  41. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms (2017)

    Google Scholar 

  42. Sechen, C.M., Sangiovanni-Vincentelli, A.L.: TimberWolf3.2: a new standard cell placement and global routing package. In: DAC, pp. 432–439. IEEE Computer Society Press (1986). https://doi.org/10.1145/318013.318083

  43. Shahookar, K., Mazumder, P.: VLSI cell placement techniques. ACM Comput. Surv. 23(2), 143–220 (1991). https://doi.org/10.1145/103724.103725

    Article  Google Scholar 

  44. Silver D., H.A.M.C.: Mastering the game of go with deep neural networks and tree search. Nature (2016)

    Google Scholar 

  45. Spindler, P., Schlichtmann, U., Johannes, F.M.: Kraftwerk2-A fast force-directed quadratic placement approach using an accurate net model. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 27(8), 1398–1411 (2008). https://doi.org/10.1109/TCAD.2008.925783

    Article  Google Scholar 

  46. Tsay, R.S., Kuh, E., Hsu, C.P.: Proud: A Fast Sea-of-Gates Placement Algorithm, pp. 318–323 (1988)

    Google Scholar 

  47. Viswanathan, N., Nam, G.J., Alpert, C., Villarrubia, P., Ren, H., Chu, C.: RQL: global placement via relaxed quadratic spreading and linearization. In: Proceedings of Design Automation Conference, pp. 453–458 (2007). https://doi.org/10.1145/1278480.1278599

  48. Viswanathan, N., Pan, M., Chu, C.: FastPlace: An Efficient Multilevel Force-Directed Placement Algorithm, pp. 193–228. Springer, Berlin (2007). https://doi.org/10.1007/978-0-387-68739-1_8

  49. William, N., Ross, D., Lu, S.: Non-linear optimization system and method for wire length and delay optimization for an automatic electric circuit placer. In: Patent US6301693B1 (2001)

    Google Scholar 

  50. Williams, R.: Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Mach Learn (1992)

    Google Scholar 

  51. Zhou, Y., Roy, S., Abdolrashidi, A., Wong, D., Ma, P.C., Xu, Q., Zhong, M., Liu, H., Goldie, A., Mirhoseini, A., Laudon, J.: GDP: Generalized Device Placement for Dataflow Graphs (2019)

    Google Scholar 

  52. Zhou, Y., Roy, S., Abdolrashidi, A., Wong, D., Ma, P., Xu, Q., Liu, H.. Phothilimtha, M. P., Wang, S., Goldie, A., Mirhoseini, A., & Laudon, J.: Transferable graph optimizers for ML compilers. In: Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  53. Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. In: Proceedings of International Conference on Learning Representations (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Google Brain, Mountain View, CA, USA

    Anna Goldie & Azalia Mirhoseini

Authors
  1. Anna Goldie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Azalia Mirhoseini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Goldie .

Editor information

Editors and Affiliations

  1. NVIDIA, Austin, TX, USA

    Haoxing Ren

  2. Texas A&M University, College Station, TX, USA

    Jiang Hu

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Goldie, A., Mirhoseini, A. (2022). RL for Placement and Partitioning. In: Ren, H., Hu, J. (eds) Machine Learning Applications in Electronic Design Automation. Springer, Cham. https://doi.org/10.1007/978-3-031-13074-8_8

Download citation

Publish with us

Policies and ethics

Search

Navigation