Matching-based preprocessing algorithms to the solution of saddle-point problems in large-scale nonconvex interior-point optimization

  • Olaf Schenk
  • Andreas Wächter
  • Michael Hagemann


Interior-point methods are among the most efficient approaches for solving large-scale nonlinear programming problems. At the core of these methods, highly ill-conditioned symmetric saddle-point problems have to be solved. We present combinatorial methods to preprocess these matrices in order to establish more favorable numerical properties for the subsequent factorization. Our approach is based on symmetric weighted matchings and is used in a sparse direct LDL T factorization method where the pivoting is restricted to static supernode data structures. In addition, we will dynamically expand the supernode data structure in cases where additional fill-in helps to select better numerical pivot elements. This technique can be seen as an alternative to the more traditional threshold pivoting techniques. We demonstrate the competitiveness of this approach within an interior-point method on a large set of test problems from the CUTE and COPS sets, as well as large optimal control problems based on partial differential equations. The largest nonlinear optimization problem solved has more than 12 million variables and 6 million constraints.


Nonconvex nonlinear programming Interior-point method Saddle-point problem Numerical linear algebra Maximum weight matching 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Benson, H.Y.: AMPL formulation of CUTE models. See
  2. 2.
    Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1–137 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Bergamaschi, L., Gondzio, J., Zilli, G.: Preconditioning indefinite systems in interior point methods for optimization. Comput. Optim. Appl. 28, 149–171 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Bondarenko, A.S., Bortz, D.M., Moré, J.J.: COPS: Large-scale nonlinearly constrained optimization problems. Technical Report ANL/MCS-TM-237, Argonne National Laboratory, Argonne, USA (1998, revised October 1999) Google Scholar
  5. 5.
    Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.L.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Software 21, 123–160 (1995) zbMATHCrossRefGoogle Scholar
  6. 6.
    Bunch, J.R., Kaufman, L.: Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput. 31, 163–179 (1977) zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Byrd, R.H., Hribar, M.E., Nocedal, J.: An interior point algorithm for large-scale nonlinear programming. SIAM J. Optim. 9, 877–900 (1999) zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Duff, I.S.: Algorithm 575: permutations for a zero-free diagonal [F1]. ACM Trans. Math. Software 7, 387–390 (1981) CrossRefMathSciNetGoogle Scholar
  10. 10.
    Duff, I.S.: MA57—A new code for the solution of sparse symmetric definite and indefinite systems. ACM Trans. Math. Software 30(2), 118–144 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Duff, I.S., Gilbert, J.R.: Maximum-weighted matching and block pivoting for symmetric indefinite systems. In: Abstract book of Householder Symposium XV, pp. 73–75 (17–21 June 2002) Google Scholar
  12. 12.
    Duff, I.S., Koster, J.: The design and use of algorithms for permuting large entries to the diagonal of sparse matrices. SIAM J. Matrix Anal. Appl. 20, 889–901 (1999) zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Duff, I.S., Pralet, S.: Strategies for scaling and pivoting for sparse symmetric indefinite problems. SIAM J. Matrix Anal. Appl. 27(2), 313–340 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Duff, I.S., Reid, J.K.: The multifrontal solution of indefinite sparse symmetric linear equations. ACM Trans. Math. Software 9, 302–325 (1983) zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    El-Bakry, A.S., Tapia, R.A., Tsuchiya, T., Zhang, Y.: On the formulation and theory of the Newton interior-point method for nonlinear programming. J. Optim. Theory Appl. 89, 507–541 (1996) zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Fiacco, A.V., McCormick, G.P.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, New York (1968). Reprinted by SIAM (1990) zbMATHGoogle Scholar
  17. 17.
    Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8, 1132–1152 (1998) zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Forsgren, A., Gill, P.E., Griffin, J.D.: Iterative solution of augmented systems arising in interior methods. Technical Report NA-05-03, University of California, San Diego (2005) Google Scholar
  19. 19.
    Forsgren, A., Gill, P.E., Wright, M.H.: Interior methods for nonlinear optimization. SIAM Rev. 44, 525–597 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Modeling Language for Mathematical Programming. Thomson, Danvers (1993) Google Scholar
  21. 21.
    Gould, H.S.D.N.I.M., Schilders, W.H.A., Wathen, A.J.: On iterative methods and implicit-factorization preconditioners for regularized saddle-point systems. Technical Report RAL-TR-2005-011, Rutherford Appleton Laboratory (2005). SIMAX (to appear) Google Scholar
  22. 22.
    Gould, N.I.M., Hu, Y., Scott, J.A.: A numerical evaluation of sparse direct solvers for the solution of large sparse, symmetric linear systems of equations. Technical Report RAL-TR-2005-005, Rutherford Appleton Laboratory (2005, to appear) Google Scholar
  23. 23.
    Gould, N.I.M., Orban, D., Sartenaer, A., Toint, P.L.: Superlinear convergence of primal–dual interior point algorithms for nonlinear programming. SIAM J. Optim. 11, 974–1002 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Gupta, A., Ying, L.: On algorithms for finding maximum matchings in bipartite graphs. Technical Report RC 21576 (97320), IBM T.J. Watson Research Center, Yorktown Heights (25 October 1999) Google Scholar
  25. 25.
    Hagemann, M., Schenk, O.: Weighted matchings for preconditioning symmetric indefinite linear systems. SIAM J. Sci. Comput. 28, 403–420 (2006) zbMATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20, 359–392 (1998) CrossRefMathSciNetGoogle Scholar
  27. 27.
    Liegmann, A.: Efficient solution of large sparse linear systems. Ph.D. thesis, ETH Zürich (1995) Google Scholar
  28. 28.
    Maurer, H., Mittelmann, H.D.: Optimization techniques for solving elliptic control problems with control and state constraints. part 1: Boundary control. Comput. Optim. Appl. 16, 29–55 (2000) zbMATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    Maurer, H., Mittelmann, H.D.: Optimization techniques for solving elliptic control problems with control and state constraints. part 2: Distributed control. Comput. Optim. Appl. 18, 141–160 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Mittelmann, H.D.: AMPL models. See
  31. 31.
    Mittelmann, H.D.: Sufficient optimality for discretized parabolic and elliptic control problems. In: Hoffmann, K.-H., Hoppe, R., Schulz, V. (eds.) Fast Solution of Discretized Optimization Problems. Birkhäuser, Basel (2001) Google Scholar
  32. 32.
    Neumaier, A.: Scaling and structural condition numbers. Linear Algebra Appl. 263, 157–165 (1997) zbMATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    Ng, E., Peyton, B.: Block sparse Cholesky algorithms on advanced uniprocessor computers. SIAM J. Sci. Comput. 14, 1034–1056 (1993) zbMATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Nocedal, J., Wächter, A., Waltz, R.A.: Adaptive barrier strategies for nonlinear interior methods. Technical Report RC 23563, IBM T.J. Watson Research Center, Yorktown Heights, USA (March 2005) Google Scholar
  35. 35.
    Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (1999) zbMATHCrossRefGoogle Scholar
  36. 36.
    Olschowka, M., Neumaier, A.: A new pivoting strategy for Gaussian elimination. Linear Algebra Appl. 240, 131–151 (1996) CrossRefMathSciNetGoogle Scholar
  37. 37.
    Schenk, O., Gärtner, K.: Two-level scheduling in PARDISO: Improved scalability on shared memory multiprocessing systems. Parallel Comput. 28, 400–441 (2002) CrossRefGoogle Scholar
  38. 38.
    Schenk, O., Gärtner, K.: On fast factorization pivoting methods for symmetric indefinite systems. Electr. Trans. Numer. Anal. 23, 158–179 (2006) zbMATHGoogle Scholar
  39. 39.
    Schenk, O., Gärtner, K., Fichtner, W.: Efficient sparse LU factorization with left–right looking strategy on shared memory multiprocessors. BIT 40, 158–176 (2000) zbMATHCrossRefMathSciNetGoogle Scholar
  40. 40.
    Ulbrich, M., Ulbrich, S., Vicente, L.N.: A globally convergent primal–dual interior-point filter method for nonlinear programming. Math. Program. 100, 379–410 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    Wächter, A., Biegler, L.T.: On the implementation of a primal–dual interior-point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006) zbMATHCrossRefMathSciNetGoogle Scholar
  42. 42.
    Wächter, A., Biegler, L.T.: Line search filter methods for nonlinear programming: Motivation and global convergence. SIAM J. Optim. 16, 1–31 (2005) zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Olaf Schenk
    • 1
  • Andreas Wächter
    • 2
  • Michael Hagemann
    • 1
  1. 1.Departement of Computer ScienceUniversity of BaselBaselSwitzerland
  2. 2.IBM T.J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations