Optimization and Engineering

, Volume 17, Issue 2, pp 359–384 | Cite as

A matrix-free augmented lagrangian algorithm with application to large-scale structural design optimization

  • Sylvain Arreckx
  • Andrew LambeEmail author
  • Joaquim R. R. A. Martins
  • Dominique Orban


In many large engineering design problems, it is not computationally feasible or realistic to store Jacobians or Hessians explicitly. Matrix-free implementations of standard optimization methods—implementations that do not explicitly form Jacobians and Hessians, and possibly use quasi-Newton approximations—circumvent those restrictions, but such implementations are virtually non-existent. We develop a matrix-free augmented-Lagrangian algorithm for nonconvex problems with both equality and inequality constraints. Our implementation is developed in the Python language, is available as an open-source package, and allows for approximating Hessian and Jacobian information.We show that our approach solves problems from the CUTEr and COPS test sets in a comparable number of iterations to state-of-the-art solvers. We report numerical results on a structural design problem that is typical in aircraft wing design optimization. The matrix-free approach makes solving problems with thousands of design variables and constraints tractable, even when function and gradient evaluations are costly.


Large-scale optimization Matrix-free optimization Structural optimization PDE-constrained optimization Augmented Lagrangian 



We would like to thank Graeme J. Kennedy for his assistance in setting up the structural design problem shown in this work. The computations for that problem were performed on the General Purpose Cluster supercomputer at the SciNet HPC Consortium.


  1. Andreani R, Birgin EG, Martínez JM, Schuverdt ML (2008) On augmented Lagrangian methods with general lower-level constraints. SIAM J Optim 18(4):1286–1309. doi: 10.1137/060654797 MathSciNetCrossRefzbMATHGoogle Scholar
  2. Argyris. H (1954) Energy theorems and structural analysis: a generalized discourse with applications on energy principles of structural analysis including the effects of temperature and non-linear stress-strain relations. Aircr Eng AerospTechnol 26(10):347–356. doi: 10.1108/eb032482 CrossRefGoogle Scholar
  3. Arioli M, Orban D (2013) Iterative methods for symmetric quasi-definite linear systems—Part I: theory. Cahier du GERAD G-2013-32, GERAD, Montréal, QC, CanadaGoogle Scholar
  4. Armand P, Benoist J, Orban D (2012) From global to local convergence of interior methods for nonlinear optimization. Optim Methods Softw 28(5):1051–1080. doi: 10.1080/10556788.2012.668905 MathSciNetCrossRefzbMATHGoogle Scholar
  5. Arreckx S, Orban D (2015) A regularized SQP method for degenerate equality-constrained optimization. Technical report, GERAD, Montréal, QC, Canada In preparationGoogle Scholar
  6. Biros G, Ghattas O (2005) Parallel Lagrange–Newton–Krylov–Schur methods for PDE-constrained optimization. Part I: the Krylov–Schur solver. SIAM J Sci Comput 27(2):687–713. doi: 10.1137/S106482750241565X MathSciNetCrossRefzbMATHGoogle Scholar
  7. Bertsekas DP (1982) Constrained optimization and Lagrange multiplier methods. Academic Press, New YorkzbMATHGoogle Scholar
  8. Byrd RH, Nocedal J, Waltz RA (2006) KNITRO: an integrated package for nonlinear optimization. In: di Pillo G, Waltz RA (eds) Large-scale nonlinear optimization. Springer Verlag, Heidelberg, pp 35–39CrossRefGoogle Scholar
  9. Conn AR, Gould NIM, Toint PhL (1992) Lancelot: a fortran package for large-scale nonlinear optimization (release A), 1st edn. Springer Publishing Company, Incorporated, New YorkCrossRefzbMATHGoogle Scholar
  10. Conn AR, Vicente LN, Visweswariah C (1999) Two-step algorithms for nonlinear optimization with structured applications. SIAM J Optim 9(4):924–947. doi: 10.1137/S1052623498334396 MathSciNetCrossRefzbMATHGoogle Scholar
  11. Conn AR, Gould NIM, Toint PhL (2000) Trust-region methods. Society for Industrial and Applied Mathematics, PhiladelphiaCrossRefzbMATHGoogle Scholar
  12. De Simone V, di Serafino D (2014) A matrix-free approach to build band preconditioners for large-scale bound-constrained optimization. J Comput Appl Math 268:82–92. doi: 10.1016/ MathSciNetCrossRefzbMATHGoogle Scholar
  13. Dener A, Kenway GK, Lyu Z, Hicken JE, Martins JRRA (2015) Comparison of inexact- and quasi-newton algorithms for aerodynamic shape optimization. In 53rd AIAA Aerospace Sciences MeetingGoogle Scholar
  14. Dennis Jr J, Walker H (1981) Convergence theorems for least-change secant update methods. SIAM J Numer Anal 18(6):949–987. doi: 10.1137/0718067 MathSciNetCrossRefzbMATHGoogle Scholar
  15. Dolan ED, Moré JJ, Munson TS (2004) Benchmarking optimization software with COPS 3.0. Technical Report ANL/MCS-TM-273, Argonne national labaratory, Mathematics and Computer Science divisionGoogle Scholar
  16. Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program 91(2):201–213. doi: 10.1007/s101070100263 MathSciNetCrossRefzbMATHGoogle Scholar
  17. Gawlik E, Munson T, Sarich J, Wild SM (2012) The TAO linearly constrained augmented Lagrangian method for PDE-constrained optimization. Preprint ANL/MCS-P2003-0112, Mathematics and Computer Science Division, January 2012. URL:
  18. Gill PE, Murray W, Saunders MA (2002) SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J Optim 12(4):979–1006. doi: 10.1137/S1052623499350013 MathSciNetCrossRefzbMATHGoogle Scholar
  19. Gill P, Robinson D (2013) A globally convergent stabilized sqp method. SIAM J Optim 23(4):1983–2010. doi: 10.1137/120882913 MathSciNetCrossRefzbMATHGoogle Scholar
  20. Gould N I M, Hribar M E, Nocedal J (2001) On the solution of equality constrained quadratic problems arising in optimization. SIAM J Sci Comput 23(4):1375–1394. doi: 10.1137/S1064827598345667 MathSciNetCrossRefzbMATHGoogle Scholar
  21. Gould NIM, Orban D, Toint PL (2003) \(\sf CUTEr \), a constrained and unconstrained testing environment, revisited. ACM Trans Math Softw 28(4):373–394. doi: 10.1145/962437.962439 MathSciNetCrossRefzbMATHGoogle Scholar
  22. Greif C, Moulding E, Orban D (2014) Bounds on the eigenvalues of block matrices arising from interior-point methods. SIAM J Optim 24(1):49–83. doi: 10.1137/120890600 MathSciNetCrossRefzbMATHGoogle Scholar
  23. Griewank A, Walther A (2002) On constrained optimization by adjoint based quasi-Newton methods. Optim Methods Softw 17:869–889. doi: 10.1080/1055678021000060829 MathSciNetCrossRefzbMATHGoogle Scholar
  24. Haftka R T, Kamat M P (1989) Simultaneous nonlinear structural analysis and design. Comput Mech 4:409–416. doi: 10.1007/BF00293046 CrossRefzbMATHGoogle Scholar
  25. Hicken JE (2014) Inexact Hessian-vector products in reduced-space differential-equation constrained optimization. Optim Eng 15:575–608. doi: 10.1007/s11081-014-9258-6 MathSciNetCrossRefGoogle Scholar
  26. Kennedy GJ, Martins JRRA (2014) A parallel finite-element framework for large-scale gradient-based design optimization of high-performance structures. Finite Elements Anal Des 87: 56–73, September 2014. ISSN 0168874X. doi: 10.1016/j.finel.2014.04.011. URL
  27. Kennedy GJ, Martins JRRA (2015) A parallel aerostructural optimization framework for aircraft design studies. Struct Multidiscipl Optim (In press). doi: 10.1007/s00158-014-1108-9
  28. Kennedy G J, Martins J RR A (2013) laminate parametrization technique for discrete ply angle problems with manufacturing constraints. Struct Multidiscipl Optim 48(2):379–393. doi: 10.1007/s00158-013-0906-9 MathSciNetCrossRefGoogle Scholar
  29. Kenway GKW, Kennedy GJ, Martins JRRA (2014) Scalable parallel approach for high-fidelity steady-state aeroelastic analysis and derivative computations. AIAA J 52(5):935–951. doi: 10.2514/1.J052255 CrossRefGoogle Scholar
  30. Kenway GKW, Martins JRRA (2014) Multipoint high-fidelity aerostructural optimization of a transport aircraft configuration. J Aircraft 51(1):144–160. doi: 10.2514/1.C032150 CrossRefGoogle Scholar
  31. Lin C-J, Moré J J (1998) Newton’s method for large bound-constrained optimization problems. SIAM J Optim 9:1100–1127. doi: 10.1137/S1052623498345075 CrossRefzbMATHGoogle Scholar
  32. Lyu Z (2014) Aerodynamic design optimization studies of a blended-wing-body aircraft. J Aircraft 51(5):1604–1617CrossRefGoogle Scholar
  33. Lyu Z, Kenway GK, Martins JRRA (2015) Aerodynamic shape optimization studies on the common research model wing benchmark. AIAA J (In press). doi: 10.2514/1.J053318
  34. Martínez HJ (1988) Local and superlinear convergence of structured secant methods for the convex class. Technical report, Rice University, Houston, USAGoogle Scholar
  35. Martins JRRA, Lambe AB (2013) Multidisciplinary design optimization: a survey of architectures. AIAA J 51(9):2049–2075. doi: 10.2514/1.J051895 CrossRefGoogle Scholar
  36. Moré JJ, Toraldo G (1989) Algorithms for bound constrained quadratic programming problems. Numer Math 55:377–400. doi: 10.1007/BF01396045 MathSciNetCrossRefzbMATHGoogle Scholar
  37. Moré JJ, Toraldo G (1991) On the solution of large quadratic programming problems with bound constraints. SIAM J Optim 1(1):93–113. doi: 10.1137/0801008 MathSciNetCrossRefzbMATHGoogle Scholar
  38. Munson T, Sarich J, Wild S, Benson S, McInnes LC (2012) TAO 2.0 Users Manual. Technical Report ANL/MCS-TM-322, Mathematics and Computer Science Division, Argonne National Laboratory.
  39. Murtagh BA, Saunders MA (2003) \(\sf MINOS \) 5.51 user’s guide. Technical Report SOL 83-20R, Stanford University, Stanford, California, USAGoogle Scholar
  40. Murtagh BA, Saunders MA (1978) Large-scale linearly constrained optimization. Math Program 14(1):41–72. doi: 10.1007/BF01588950 MathSciNetCrossRefzbMATHGoogle Scholar
  41. Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, New YorkzbMATHGoogle Scholar
  42. Nocedal J, Yuan Y (1998) Combining trust region and line search techniques. Technical report, Advances in Nonlinear Programming, (Kluwer)Google Scholar
  43. Orban D (2014) NLPy—a large-scale optimization toolkit in Python. Cahier du GERAD G-2014-xx, GERAD, Montréal, QC, Canada. In preparationGoogle Scholar
  44. Perez RE, Jansen PW, Martins JRRA (2012) pyOpt: a Python-based object-oriented framework for nonlinear constrained optimization. Struct Multidiscipl Optim 45(1):101–118. doi: 10.1007/s00158-011-0666-3 MathSciNetCrossRefzbMATHGoogle Scholar
  45. Poon NMK, Martins JRRA (2007) An adaptive approach to constraint aggregation using adjoint sensitivity analysis. Struct Multidiscipl Optim 34:61–73. doi: 10.1007/s00158-006-0061-7 CrossRefGoogle Scholar
  46. Rockafellar RT (1973) The multiplier method of hestenes and powell applied to convex programming. J Optim Theory Appl 12(6): 555–562. ISSN 0022-3239. doi: 10.1007/BF00934777
  47. Schlenkrich S, Griewank A, Walther A (2010) On the local convergence of adjoint Broyden methods. Math Program 121:221–247. doi: 10.1007/s10107-008-0232-y MathSciNetCrossRefzbMATHGoogle Scholar
  48. Schmit LA (1960) Structural design by systematic synthesis. In 2nd Conference on Electronic Computation, ASCE, New York, NY, pp 105–132Google Scholar
  49. Toint PL (1997) Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints. Math Program 77(3):69–94. doi: 10.1007/BF02614518 MathSciNetCrossRefzbMATHGoogle Scholar
  50. Turner MJ, Clough RW, Martin HC, Topp LJ (1956) Stiffness and deflection analysis of complex structures. J Aeronaut Sci 23(9):805–823CrossRefzbMATHGoogle Scholar
  51. Wächter A, Biegler LT (2006) On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math Program 106:25–57. doi: 10.1007/s10107-004-0559-y MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Sylvain Arreckx
    • 1
    • 2
  • Andrew Lambe
    • 3
    Email author
  • Joaquim R. R. A. Martins
    • 4
  • Dominique Orban
    • 1
    • 2
  1. 1.GERADMontréalCanada
  2. 2.Department of Mathematics and Industrial EngineeringÉcole PolytechniqueMontréalCanada
  3. 3.University of Toronto Institute for Aerospace StudiesTorontoCanada
  4. 4.Department of Aerospace EngineeringUniversity of MichiganAnn ArborUSA

Personalised recommendations