Mathematical Programming Computation

, Volume 8, Issue 4, pp 435–459 | Cite as

An SR1/BFGS SQP algorithm for nonconvex nonlinear programs with block-diagonal Hessian matrix

  • Dennis JankaEmail author
  • Christian Kirches
  • Sebastian Sager
  • Andreas Wächter
Full Length Paper


We present a quasi-Newton sequential quadratic programming (SQP) algorithm for nonlinear programs in which the Hessian of the Lagrangian function is block-diagonal. Problems with this characteristic frequently arise in the context of optimal control; for example, when a direct multiple shooting parametrization is used. In this article, we describe an implementation of a filter line-search SQP method that computes search directions using an active-set quadratic programming (QP) solver. To take advantage of the block-diagonal structure of the Hessian matrix, each block is approximated separately by quasi-Newton updates. For nonconvex instances, that arise, for example, in optimum experimental design control problems, these blocks are often found to be indefinite. In that case, the block-BFGS quasi-Newton update can lead to poor convergence. The novel aspect in this work is the use of SR1 updates in place of BFGS approximations whenever possible. The resulting indefinite QPs necessitate an inertia control mechanism within the sparse Schur-complement factorization that is carried out by the active-set QP solver. This permits an adaptive selection of the Hessian approximation that guarantees sufficient progress towards a stationary point of the problem. Numerical results demonstrate that the proposed approach reduces the number of SQP iterations and CPU time required for the solution of a set of optimal control problems.


Quasi-Newton Sequential quadratic programming Direct methods for optimal control Optimum experimental design 

Mathematics Subject Classification

49M37 90C20 90C30 90C53 90C55 



The authors thank two anonymous referees and an anonymous technical editor for their constructive and helpful comments.

D. Janka and C. Kirches were supported by DFG Graduate School 220 (Heidelberg Graduate School of Mathematical and Computational Methods for the Sciences) funded by the German Excellence Initiative. D. Janka was supported by BASF SE within the junior research group experimental design. C. Kirches and S. Sager were supported by the German Federal Ministry of Education and Research program “Mathematics for Innovations in Industry and Service 2013–2016”, Grant No. 05M2013-GOSSIP. Financial support by the European Union within the 7th Framework Programme under Grant Agreement No 611909 is gratefully acknowledged. A. Wächter was supported in part by the National Science Foundation Grant DMS-1216920. Parts of this work were carried out during an appointment of D. Janka to Northwestern University.


  1. 1.
    Amestoy, P., Dollar, H.S., Reid, J.K., Scott, J.A.: An approximate minimum degree algorithm for matrices with dense rows. Technical Report RAL-TR-2007-020, Rutherford Appleton Laboratory (2007)Google Scholar
  2. 2.
    Bauer, I., Bock, H.G., Körkel, S., Schlöder, J.P.: Numerical methods for initial value problems and derivative generation for DAE models with application to optimum experimental design of chemical processes. Sci. Comput. Chem. Eng. II(2), 282–289 (1999)Google Scholar
  3. 3.
    Bauer, I., Bock, H.G., Schlöder, J.P.: DAESOL—a BDF-code for the numerical solution of differential algebraic equations. Preprint, IWR der Universität Heidelberg, SFB 359 (1999)Google Scholar
  4. 4.
    Bischof, C., Khademi, P., Mauer, A., Carle, A.: ADIFOR 2.0: automatic differentiation of fortran 77 programs. IEEE Comput. Sci. Eng. 3(3), 18–32 (1996)CrossRefGoogle Scholar
  5. 5.
    Bisschop, J., Meeraus, A.: Matrix augmentation and partitioning in the updating of the basis inverse. Math. Program. 13(1), 241–254 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bock, H.G.: Randwertproblemmethoden zur Parameteridentifizierung in Systemen nichtlinearer Differentialgleichungen, volume 183 of Bonner Mathematische Schriften. Universität Bonn, Bonn (1987)Google Scholar
  7. 7.
    Bock, H.G., Plitt, K.J.: A multiple shooting algorithm for direct solution of optimal control problems. In: Proceedings of the 9th IFAC World Congress, pages 242–247, Budapest, 1984. Pergamon Press. Available at
  8. 8.
    Bryson, A.E.: Applied Optimal Control: Optimization, Estimation and Control. CRC Press, Boca Raton (1975)Google Scholar
  9. 9.
    Byrd, R.H., Khalfan, H.F., Schnabel, R.B.: Analysis of a symmetric rank-one trust region method. SIAM J. Optim. 6(4), 1025–1039 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Conn, A.R., Gould, N.I.M., Toint, PhL: Testing a class of methods for solving minimization problems with simple bounds on the variables. Math. Comput. 50(182), 399–430 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Conn, A.R., Gould, N.I.M., Toint, PhL: Convergence of quasi-Newton matrices generated by the symmetric rank one update. Math. Program. 50(1–3), 177–195 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Conn, A.R., Gould, N.I.M., Toint, PhL: LANCELOT: A Fortran Package for Large-scale Nonlinear Optimization (Release A). Volume 17 of Springer Series in Computational Mathematics. Springer, Heidelberg (1992)CrossRefzbMATHGoogle Scholar
  13. 13.
    Contreras, M., Tapia, R.A.: Sizing the BFGS and DFP updates: numerical study. J. Optim. Theory Appl. 78(1), 93–108 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Cuthrell, J.E., Biegler, L.T.: Simultaneous optimization and solution methods for batch reactor control profiles. Comput. Chem. Eng. 13(1), 49–62 (1989)CrossRefGoogle Scholar
  15. 15.
    Diehl, M., Bock, H.G., Kostina, E.: An approximation technique for robust nonlinear optimization. Math. Program. 107, 213–230 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Dolan, E.D., Moré, J.J., Munson, T.S.: Benchmarking optimization software with COPS 3.0. Argonne National Laboratory Technical Report ANL/MCS-TM-273 (2004)Google Scholar
  18. 18.
    Duff, I.S.: MA57—a code for the solution of sparse symmetric definite and indefinite systems. ACM Trans. Math. Softw. 30(2), 118–144 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Ferreau, H.J., Bock, H.G., Diehl, M.: An online active set strategy to overcome the limitations of explicit MPC. Int. J. Robust Nonlinear Control 18(8), 816–830 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Ferreau, H.J., Kirches, C., Potschka, A., Bock, H.G., Diehl, M.: qpOASES 3.0: a parametric active-set algorithm for quadratic programming. Math. Program. Comput. 6(4), 327–363 (2014)Google Scholar
  21. 21.
    Fletcher, R.: Stable reduced hessian updates for indefinite quadratic programming. Math. Program. 87(2), 251–264 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Forbes, J.: Model structure and adjustable parameter selection for operations optimizations. PhD thesis, McMaster University, Hamilton, Canada (1994)Google Scholar
  23. 23.
    Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: A Schur-complement method for sparse quadratic programming. Technical report, Stanford Univ., CA (USA). Systems Optimization Lab. (1987)Google Scholar
  24. 24.
    Gill, P.E., Wong, E.: Methods for convex and general quadratic programming. Technical report, University of California at San Diego (2013)Google Scholar
  25. 25.
    Gould, N.I.M., Toint, PhL: An iterative working-set method for large-scale nonconvex quadratic programming. Appl. Numer. Math. 43(1), 109–128 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Griewank, A., Toint, PhL: Local convergence analysis for partitioned quasi-Newton updates. Numerische Mathematik 39(3), 429–448 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Griewank, A., Toint, PhL: Partitioned variable metric updates for large structured optimization problems. Numerische Mathematik 39(1), 119–137 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Janka, D.: Optimum experimental design and multiple shooting. Diplomarbeit, Universität Heidelberg, Heidelberg (2010)Google Scholar
  29. 29.
    Janka, D.: Sequential quadratic programming with indefinite Hessian approximations for nonlinear optimum experimental design for parameter estimation in differential-algebraic equations. PhD thesis, Ruprecht-Karls-Universität Heidelberg, 2015. Available at
  30. 30.
    Khalfan, H.F., Byrd, R.H., Schnabel, R.B.: A theoretical and experimental study of the symmetric rank-one update. SIAM J. Optim. 3(1), 1–24 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Körkel, S.: Numerische Methoden für Optimale Versuchsplanungsprobleme bei nichtlinearen DAE-Modellen. PhD thesis, Universität Heidelberg (2002)Google Scholar
  32. 32.
    Körkel, S., Potschka, A., Bock, H.G., Sager, S.: A multiple shooting formulation for optimum experimental design. (2008). (submitted to Mathematical Programming, under review)Google Scholar
  33. 33.
    Leineweber, D.B., Bauer, I., Schäfer, A.A.S., Bock, H.G., Schlöder, J.P.: An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization (parts I and II). Comput. Chem. Eng. 27, 157–174 (2003)CrossRefGoogle Scholar
  34. 34.
    Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, Berlin (2006). ISBN 0-387-30303-0 (hardcover)Google Scholar
  35. 35.
    Oren, S.S., Luenberger, D.G.: Self-scaling variable metric (SSVM) algorithms part i: criteria and sufficient conditions for scaling a class of algorithms. Manag. Sci. 20(5), 845–862 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Pardalos, P.M., Vavasis, S.A.: Quadratic programming with one negative eigenvalue is NP-hard. J. Glob. Optim. 1(1), 15–22 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Plitt, K.J.: Ein superlinear konvergentes Mehrzielverfahren zur direkten Berechnung beschränkter optimaler Steuerungen. Diplomarbeit. Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn (1981)Google Scholar
  38. 38.
    Powell, M.J.D.: The convergence of variable metric methods for non-linearly constrained optimization calculations. Nonlinear Program. 3, 27–63 (1978)Google Scholar
  39. 39.
    Pukelsheim, F.: Optimal Design of Experiments. Classics in Applied Mathematics 50. SIAM (2006). ISBN 978-0-898716-04-7Google Scholar
  40. 40.
    Sager, S.: On the Integration of Optimization Approaches for Mixed-Integer Nonlinear Optimal Control. Universität Heidelberg, August (2011). HabilitationsschriftGoogle Scholar
  41. 41.
    Sager, S.: Sampling decisions in optimum experimental design in the light of Pontryagin’s maximum principle. SIAM J. Control Optim. 51(4), 3181–3207 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Sager, S., Bock, H.G., Diehl, M., Reinelt, G., Schlöder, J.P.: Numerical methods for optimal control with binary control functions applied to a Lotka–Volterra type fishing problem. In: Seeger, A. (ed.) Recent Advances in Optimization, Volume 563 of Lectures Notes in Economics and Mathematical Systems, pp. 269–289. Springer, Heidelberg (2009). ISBN 978-3-5402-8257-0Google Scholar
  43. 43.
    Vanderbei, R.J., Shanno, D.F.: An interior-point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13, 231–252 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Vassiliadis, V.S., Sargent, R.W.H., Pantelides, C.C.: Solution of a class of multistage dynamic optimization problems. Parts 1. and 2. Indus. Eng. Chem. Res. 10(33), 2111–2133 (1994)CrossRefGoogle Scholar
  45. 45.
    Wächter, A., Biegler, L.T.: Line search filter methods for nonlinear programming: motivation and global convergence. SIAM J. Optim. 16(1), 1–31 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Wächter, A., Biegler, L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and The Mathematical Programming Society 2016

Authors and Affiliations

  • Dennis Janka
    • 1
    Email author
  • Christian Kirches
    • 1
  • Sebastian Sager
    • 2
  • Andreas Wächter
    • 3
  1. 1.Interdisciplinary Center for Scientific Computing (IWR)Heidelberg UniversityHeidelbergGermany
  2. 2.Institute for Mathematical OptimizationOtto-von-Guericke-University MagdeburgMagdeburgGermany
  3. 3.Department of Industrial Engineering and Management SciencesNorthwestern UniversityEvanstonUSA

Personalised recommendations