Skip to main content
SpringerLink
Log in

A graph-based methodology for constructing computational models that automates adjoint-based sensitivity analysis

  • Research
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

The adjoint method provides an efficient way to compute sensitivities for system models with a large number of inputs. However, implementing the adjoint method requires significant effort that limits its use. The effort is exacerbated in large-scale multidisciplinary design optimization. We propose the adoption of a three-stage compiler as the method for constructing computational models for large-scale multidisciplinary design optimization to enable accurate and efficient adjoint sensitivity analysis. We develop a new modeling language called the Computational System Design Language that provides an appropriate input to the compiler front end that works well with multidisciplinary models. This paper describes the three-stage compiler methodology and the Computational System Design Language. The proposed solution uses a graph representation of the numerical model to automatically generate a computational model that computes adjoint-based sensitivities for use within an optimization framework. For two engineering models, this approach reduces the amount of user code by a factor of approximately two compared to their original implementations, without a measurable increase in computation time. This paper also includes a best-case complexity analysis that is built into the compiler implementation to allow users to estimate the memory required to evaluate a computational model and its derivatives, which is independent of the compiler back end that ultimately generates the computational model. Future compiler implementations are expected to approach the theoretical best-case memory cost and improve run time performance for both model evaluation and derivative computation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

No datasets were generated or analysed during the current study.

Notes

  1. Another AD library that supports implicit differentiation in Julia is available at https://gdalle.github.io/ImplicitDifferentiation.jl/.

  2. The term “graph-based” here refers to a property of languages, and not a property of the methodology presented in this paper.

  3. There are two modes when solving the UDE: forward mode and reverse mode. Forward mode corresponds to using the direct method and reverse mode corresponds to using the adjoint method.

  4. This is the same as the concept of a model within the Problem class in OpenMDAO.

  5. The derivative implementation for external code may be inexact due to limitations in how the external code was implemented.

  6. Another AD library that supports implicit differentiation is available at https://gdalle.github.io/ImplicitDifferentiation.jl/.

  7. This assumes the existence of a compiler back end that can support code generation for a particular architecture, which is not the case at the time of writing.

  8. Back ends that generate code in these languages do not currently exist, but can be developed to use CSDL GraphRepresentation objects generated by the CSDL compiler front end.

  9. Execution of individual vectorized operations may be further parallelized, reducing computation time by more than what is shown in Fig. 8.

References

  • Åkesson J (2008) Optimica—an extension of modelica supporting dynamic optimization. In: 6th international modelica conference, pp 57–66

  • Alnæs MS, Mardal K-A (2010) On the efficiency of symbolic computations combined with code generation for finite element methods. ACM Trans Math Softw 37(1):1–26

    Article  MathSciNet  Google Scholar 

  • Alnæs MS, Mardal K-A (2012) SYFI and SFC: symbolic finite elements and form compilation. Springer, New York

    Google Scholar 

  • Alnæs MS, Logg A, Ølgaard KB, Rognes ME, Wells GN (2014) Unified form language: a domain-specific language for weak formulations of partial differential equations. ACM Trans Math Softw 40(2):1–37

    Article  MathSciNet  Google Scholar 

  • Baydin AG, Pearlmutter BA, Radul AA, Siskind JM (2018) Automatic differentiation in machine learning: a survey. J March Learn Res 18:1–43

    Google Scholar 

  • Bezanson J, Edelman A, Karpinski S, Shah VB (2017) Julia: a fresh approach to numerical computing. SIAM Rev 59(1):65–98

    Article  MathSciNet  Google Scholar 

  • Bisschop J, Meeraus A (1982) On the development of a general algebraic modeling system in a strategic planning environment. Springer, New York

    Book  Google Scholar 

  • Blondel M, Berthet Q, Cuturi M, Frostig R, Hoyer S, Llinares-López F, Pedregosa F, Vert JP (2022) Efficient and modular implicit differentiation. arXiv:2105.15183

  • Bradbury J, Frostig R, Hawkins P, Johnson MJ, Leary C, Maclaurin D, Necula G, Paszke A, VanderPlas J, Wanderman-Milne S, Zhang Q (2018) JAX: composable transformations of Python+NumPy programs. http://github.com/google/jax

  • Bussieck MR, Meeraus A (2004) General algebraic modeling system (GAMs). Springer, New York

    Book  Google Scholar 

  • Bynum ML, Hackebeil GA, Hart WE, Laird CD, Nicholson BL, Siirola JD, Watson JP, Woodruff DL (2021) Pyomo-optimization modeling in python, vol 67. Springer, Berlin

    Book  Google Scholar 

  • Carpenter B, Hoffman MD, Brubaker M, Lee D, Li P, Betancourt M (2015) The Stan math library: reverse-mode automatic differentiation in c++. arXiv preprint arXiv:1509.07164

  • Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, Brubaker MA, Guo J, Li P, Riddell A (2017) Stan: a probabilistic programming language. J Stat Softw 76:1

    Article  Google Scholar 

  • Chauhan SS, Martins JRRA (2018) Low-fidelity aerostructural optimization of aircraft wings with a simplified wingbox model using OpenAeroStruct. Springer, New York

    Google Scholar 

  • Cheng Z, Zhao S, Scotzniovsky L, Rodriguez G, Mi C, Hwang JT (2023) A differentiable method for low-fidelity analysis of permanent-magnet synchronous motors. In: AIAA SCITECH 2023 forum, p 1091

  • Dunning I, Huchette J, Lubin M (2017) Jump: a modeling language for mathematical optimization. SIAM Rev 59(2):295–320

    Article  MathSciNet  Google Scholar 

  • Fishwick PA (1998) A taxonomy for simulation modeling based on programming language principles. IIE Trans 30(9):811–820

    Article  Google Scholar 

  • Fourer R, Gay DM, Kernighan BW (1990) A modeling language for mathematical programming. Manag Sci 36(5):519–554

    Article  Google Scholar 

  • Fritzson P, Engelson V (1998) Modelica—a unified object-oriented language for system modeling and simulation. In: ECOOP'’98—Object-Oriented Programming: 12th European Conference Brussels, Belgium, July 20–24, 1998 Proceedings 12. Springer, Berlin, pp 67–90

  • Gray J, Moore K, Naylor B (2010) Openmdao: an open source framework for multidisciplinary analysis and optimization. In: 13th AIAA/ISSMO multidisciplinary analysis optimization conference, vol 5

  • Gray JS, Hwang JT, Martins JR, Moore KT, Naylor BA (2019) Openmdao: an open-source framework for multidisciplinary design, analysis, and optimization. Struct Multidisc Optim 59(4):1075–1104

    Article  MathSciNet  Google Scholar 

  • Ha TH, Lee K, Hwang JT (2020) Large-scale multidisciplinary optimization under uncertainty for electric vertical takeoff and landing aircraft. In: IAA Scitech 2020 forum, p 0904

  • Hart WE, Watson J-P, Woodruff DL (2011) Pyomo: modeling and solving mathematical programs in python. Math Program Comput 3(3):219–260

    Article  MathSciNet  Google Scholar 

  • Hwang JT, Martins JR (2018) A computational architecture for coupling heterogeneous numerical models and computing coupled derivatives. ACM Trans Math Softw 44(4):1–39

    Article  MathSciNet  Google Scholar 

  • Hwang JT, Ning A (2018) Large-scale multidisciplinary optimization of an electric aircraft for on-demand mobility. In: 2018 AIAA/ASCE/AHS/ASC structures, structural dynamics, and materials conference, p 1384

  • Jasa JP, Hwang JT, Martins JRRA (2018) Open-source coupled aerostructural optimization using Python. Struct Multidisc Optim 4:1815–1827

    Article  Google Scholar 

  • Kahn AB (1962) Topological sorting of large networks. Commun ACM 5(11):558–562

    Article  Google Scholar 

  • Kallrath J (2004) Modeling languages in mathematical optimization. Springer, New York

    Book  Google Scholar 

  • Kallrath J (2012) Algebraic modeling systems: modeling and solving real world optimization problems, 104. Springer, New York

    Book  Google Scholar 

  • Kirby RC, Logg A (2006) A compiler for variational forms. ACM Trans Math Softw 32(3):417–444

    Article  MathSciNet  Google Scholar 

  • Lattner C, Adve V (2004) LLVM: a compilation framework for lifelong program analysis & transformation. In: International symposium on code generation and optimization, 2004. CGO 2004, pp 75-86. IEEE

  • Logg A, Ølgaard KB, Rognes ME, Wells GN (2012) FFC: the fenics form compiler. Springer, New York

    Google Scholar 

  • Margossian CC (2019) A review of automatic differentiation and its efficient implementation. Wiley Interdiscip Rev 9(4):e1305

    Google Scholar 

  • Margossian CC, Betancourt M (2021) Efficient automatic differentiation of implicit functions. arXiv preprint arXiv:2112.14217

  • Marriage C, Martins JR (2008) Reconfigurable semi-analytic sensitivity methods and mdo architectures within the pimdo framework. In: 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, p 5956

  • Martins JR, Hwang JT (2013) Review and unification of methods for computing derivatives of multidisciplinary computational models. AIAA J 51(11):2582–2599

    Article  Google Scholar 

  • Martins JR, Sturdza P, Alonso JJ (2003) The complex-step derivative approximation. ACM Trans Math Softw 29(3):245–262

    Article  MathSciNet  Google Scholar 

  • Mernik M, Heering J, Sloane AM (2005) When and how to develop domain-specific languages. ACM Comput Surv 37(4):316–344

    Article  Google Scholar 

  • Moses W, Churavy V (2020) Instead of rewriting foreign code for machine learning, automatically synthesize fast gradients. Curran Associates Inc., New York

    Google Scholar 

  • Moses WS, Churavy V, Paehler L, Hückelheim J, Narayanan SH, Schanen M, Doerfert J (2021) Reverse-mode automatic differentiation and optimization of GPU kernels via enzyme. Association for Computing Machinery

  • Moses WS, Narayanan SH, Paehler L, Churavy V, Schanen M, Hückelheim J, Doerfert J, Hovland P (2022) Scalable automatic differentiation of multiple parallel paradigms through compiler augmentation. In: SC22: international conference for high performance computing, networking, storage and analysis, pp 1–18. IEEE

  • Ning A, McDonnell T (2023) Automating steady and unsteady adjoints: efficiently utilizing implicit and algorithmic differentiation. arXiv:2306.15243

  • Ølgaard KB, Wells GN (2010) Optimizations for quadrature representations of finite element tensors through automated code generation. ACM Trans Math Softw 37(1):1–23

    Article  MathSciNet  Google Scholar 

  • Revels J, Lubin M, Papamarkou T (2016) Forward-mode automatic differentiation in Julia. arXiv preprint arXiv:1607.07892

  • Rognes ME, Kirby RC, Logg A (2010) Efficient assembly of h(div) and h(curl) conforming finite elements. SIAM J Sci Comput 31(6):4130–4151

    Article  MathSciNet  Google Scholar 

  • Ruh ML, Hwang JT (2021) Robust modeling and optimal design of rotors using blade element momentum theory. In: AIAA Aviation 2021 forum, p 2598

  • Ruh ML, Hwang JT (2023) Fast and robust computation of optimal rotor designs using blade element momentum theory. AIAA J 61(9):4096–4111

    Article  Google Scholar 

  • Sarojini D, Ruh ML, Joshy AJ, Yan J, Ivanov AK, Scotzniovsky L, Fletcher AH, Orndorff NC, Sperry M, Gandarillas VE, Asher I (2023) Large-scale multidisciplinary design optimization of an evtol aircraft using comprehensive analysis. In: AIAA SciTech 2023 forum, p 0146

  • Sinha R, Paredis CJ, Liang V-C, Khosla PK (2001) Modeling and simulation methods for design of engineering systems. J Comput Inf Sci Eng 1(1):84–91

    Article  Google Scholar 

  • Squire W, Trapp G (1998) Using complex variables to estimate derivatives of real functions. SIAM Rev 40(1):110–112

    Article  MathSciNet  Google Scholar 

  • Viklund L, Fritzson P (1995) Objectmath-an object-oriented language and environment for symbolic and numerical processing in scientific computing. Sci Program 4(4):229–250

    Google Scholar 

  • Wang B, Sperry M, Gandarillas VE, Hwang JT (2022) Efficient uncertainty propagation through computational graph modification and automatic code generation. In: AIAA AVIATION 2022 forum, p 3997

  • Wang B, Orndorff NC, Sperry M, Hwang JT (2023) High-dimensional uncertainty quantification using graph-accelerated non-intrusive polynomial chaos and active subspace methods. In: AIAA AVIATION 2023 forum, p 4264

  • Wang B, Sperry M, Gandarillas VE, Hwang JT. Accelerating model evaluations in uncertainty propagation on tensor grids using computational graph transformations (under review)

  • Yang J (2021) Fastad: expression template-based c++ library for fast and memory-efficient automatic differentiation. arXiv preprint arXiv:2102.03681

Download references

Acknowledgements

The authors thank Marius Ruh, Jiayao Yan, and Luca Scotzionovsky for helping generate the data in Fig. 10.

Funding

This work was supported in part by the National Science Foundation under Grant no. 1936557 and NASA Grant number 80NSSC21M0070.

Author information

Authors and Affiliations

  1. Department of Mechanical and Aerospace Engineering, University of California San Diego, 9500 Gilman Dr, La Jolla, CA, 92093, USA

    Victor Gandarillas, Anugrah Jo Joshy, Mark Z. Sperry, Alexander K. Ivanov & John T. Hwang

Authors
  1. Victor Gandarillas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anugrah Jo Joshy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark Z. Sperry
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander K. Ivanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John T. Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

V.G. led the development of the language and compiler presented in the paper and wrote the manuscript. A.J. and A.L. developed the standard library for the language presented in the paper. M.S. contributed to the development of the compiler presented in the paper. J.H. supervised the research and manuscript writing. All authors reviewed the manuscript.

Corresponding author

Correspondence to Victor Gandarillas.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no Conflict of interest.

Replication of results

Detailed documentation for the Computational System Design Language (CSDL) is available at the CSDL website, https://lsdolab.github.io/csdl/. Links to the GitHub repository containing the compiler implementation, including the CSDL compiler front end and compiler middle end (https://github.com/lsdolab/csdl) along with a list of links to available back ends are also provided within the CSDL website.

Ethical approval

Not Applicable.

Informed consent

Not Applicable.

Additional information

Responsible Editor: Nestor Queipo

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gandarillas, V., Joshy, A.J., Sperry, M.Z. et al. A graph-based methodology for constructing computational models that automates adjoint-based sensitivity analysis. Struct Multidisc Optim 67, 76 (2024). https://doi.org/10.1007/s00158-024-03792-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00158-024-03792-0

Keywords

Search

Navigation