Structural and Multidisciplinary Optimization

, Volume 60, Issue 4, pp 1709–1713 | Cite as

Framework for design optimization using deep reinforcement learning

  • Kazuo YonekuraEmail author
  • Hitoshi Hattori
Brief Note


We propose a framework for design optimization using deep reinforcement learning and study its capabilities. Reinforcement learning is highly generalizable to unseen system configurations for similar optimization problems. In industrial fields, product requirements vary depending on specifications and the requirements are often similar but slightly different from each other. We utilize reinforcement learning to optimize products to meet those slightly different requirements. In the proposed framework, an agent is trained in advance and used to find the optimal solution given a set of requirements. We apply the proposed framework to optimize the airfoil angle of attack and demonstrate its generalization capabilities.


Deep reinforcement learning Design optimization Machine learning 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Abbott IH, Von Doenhoff AE (1949) Theory of wing sections including a summary of airfoil data. DoverGoogle Scholar
  2. Andrychowicz M, Denil M, Gómez S., Hoffman MW, Pfau D, Schaul T, Shillingford B, de Freitas N (2016) Learning to learn by gradient descent by gradient descent. In: Advances in Neural Information Processing Systems 29Google Scholar
  3. Kober J, Bagnell JA, Peters J (2013) Reinforcement learning in robotics: a survey. Int J Robot Res 32 (11):1238–1274CrossRefGoogle Scholar
  4. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25Google Scholar
  5. Launder B, Spalding D (1974) The numerical computation of turbulent flows. Comput Methods Appl Mech Eng 3:269–289CrossRefGoogle Scholar
  6. Li K, Malik J (2017) Learning to optimize. In: ICLR 2017 ConferenceGoogle Scholar
  7. Mack Y, Goel T, Shyy W, Haftka R (2007) Surrogate Model-Based Optimization Framework: A Case Study in Aerospace Design. Springer, Berlin, pp 323–342Google Scholar
  8. Nita K, Okita Y, Nakamata C, Kubo S, Yonekura K, Watanabe O (2014) Film cooling hole shape optimization using proper orthogonal decomposition. In: ASME Turbo expo 2014Google Scholar
  9. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830MathSciNetzbMATHGoogle Scholar
  10. Reddy G, Celani A, Sejnowski TJ, Vergassola M (2016) Learning to soar in turbulent environments. Proc Natl Acad Sci 113(33):E4877–E4884CrossRefGoogle Scholar
  11. Samad A, Kim KY (2008) Shape optimization of an axial compressor blade by multi-objective genetic algorithm. Proceedings of the Institution of Mechanical Engineers, Part A: Journal of Power and Energy 222(6):599–611Google Scholar
  12. Sutton RS, Barto AG (2018) Reinforcement learning: An introduction, 2nd edn. The MIT PressGoogle Scholar
  13. Tokui S, Oono K, Hido S, Clayton J (2015) Chainer: a next-generation open source framework for deep learning. In: Proceedings of Workshop on Machine Learning Systems in The Twenty-ninth Annual Conference on Neural Information Processing SystemsGoogle Scholar
  14. Wang DX, He L (2010) Adjoint aerodynamic design optimization for blades in multistage turbomachines—Part I: methodology and verification. J Turbomach 132(2):021,011–021,011–14CrossRefGoogle Scholar
  15. Wang GG, Shan S (2006) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129:370–380CrossRefGoogle Scholar
  16. Yonekura K, Watanabe O (2014) A shape parameterization method using principal component analysis in applications to parametric shape optimization. J Mech Des 12(121):401Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Computational and Mathematical Engineering DepartmentIHI CorporationIsogo-kuJapan

Personalised recommendations