Matching-based preprocessing algorithms to the solution of saddle-point problems in large-scale nonconvex interior-point optimization



Interior-point methods are among the most efficient approaches for solving large-scale nonlinear programming problems. At the core of these methods, highly ill-conditioned symmetric saddle-point problems have to be solved. We present combinatorial methods to preprocess these matrices in order to establish more favorable numerical properties for the subsequent factorization. Our approach is based on symmetric weighted matchings and is used in a sparse direct LDLT factorization method where the pivoting is restricted to static supernode data structures. In addition, we will dynamically expand the supernode data structure in cases where additional fill-in helps to select better numerical pivot elements. This technique can be seen as an alternative to the more traditional threshold pivoting techniques. We demonstrate the competitiveness of this approach within an interior-point method on a large set of test problems from the CUTE and COPS sets, as well as large optimal control problems based on partial differential equations. The largest nonlinear optimization problem solved has more than 12 million variables and 6 million constraints.


Nonconvex nonlinear programming Interior-point method Saddle-point problem Numerical linear algebra Maximum weight matching 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Benson, H.Y.: AMPL formulation of CUTE models. See
  2. 2.
    Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1–137 (2005) MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Bergamaschi, L., Gondzio, J., Zilli, G.: Preconditioning indefinite systems in interior point methods for optimization. Comput. Optim. Appl. 28, 149–171 (2004) MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Bondarenko, A.S., Bortz, D.M., Moré, J.J.: COPS: Large-scale nonlinearly constrained optimization problems. Technical Report ANL/MCS-TM-237, Argonne National Laboratory, Argonne, USA (1998, revised October 1999) Google Scholar
  5. 5.
    Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.L.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Software 21, 123–160 (1995) MATHCrossRefGoogle Scholar
  6. 6.
    Bunch, J.R., Kaufman, L.: Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput. 31, 163–179 (1977) MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Byrd, R.H., Hribar, M.E., Nocedal, J.: An interior point algorithm for large-scale nonlinear programming. SIAM J. Optim. 9, 877–900 (1999) MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002) MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Duff, I.S.: Algorithm 575: permutations for a zero-free diagonal [F1]. ACM Trans. Math. Software 7, 387–390 (1981) CrossRefMathSciNetGoogle Scholar
  10. 10.
    Duff, I.S.: MA57—A new code for the solution of sparse symmetric definite and indefinite systems. ACM Trans. Math. Software 30(2), 118–144 (2004) MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Duff, I.S., Gilbert, J.R.: Maximum-weighted matching and block pivoting for symmetric indefinite systems. In: Abstract book of Householder Symposium XV, pp. 73–75 (17–21 June 2002) Google Scholar
  12. 12.
    Duff, I.S., Koster, J.: The design and use of algorithms for permuting large entries to the diagonal of sparse matrices. SIAM J. Matrix Anal. Appl. 20, 889–901 (1999) MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Duff, I.S., Pralet, S.: Strategies for scaling and pivoting for sparse symmetric indefinite problems. SIAM J. Matrix Anal. Appl. 27(2), 313–340 (2005) MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Duff, I.S., Reid, J.K.: The multifrontal solution of indefinite sparse symmetric linear equations. ACM Trans. Math. Software 9, 302–325 (1983) MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    El-Bakry, A.S., Tapia, R.A., Tsuchiya, T., Zhang, Y.: On the formulation and theory of the Newton interior-point method for nonlinear programming. J. Optim. Theory Appl. 89, 507–541 (1996) MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Fiacco, A.V., McCormick, G.P.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, New York (1968). Reprinted by SIAM (1990) MATHGoogle Scholar
  17. 17.
    Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8, 1132–1152 (1998) MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Forsgren, A., Gill, P.E., Griffin, J.D.: Iterative solution of augmented systems arising in interior methods. Technical Report NA-05-03, University of California, San Diego (2005) Google Scholar
  19. 19.
    Forsgren, A., Gill, P.E., Wright, M.H.: Interior methods for nonlinear optimization. SIAM Rev. 44, 525–597 (2002) MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Modeling Language for Mathematical Programming. Thomson, Danvers (1993) Google Scholar
  21. 21.
    Gould, H.S.D.N.I.M., Schilders, W.H.A., Wathen, A.J.: On iterative methods and implicit-factorization preconditioners for regularized saddle-point systems. Technical Report RAL-TR-2005-011, Rutherford Appleton Laboratory (2005). SIMAX (to appear) Google Scholar
  22. 22.
    Gould, N.I.M., Hu, Y., Scott, J.A.: A numerical evaluation of sparse direct solvers for the solution of large sparse, symmetric linear systems of equations. Technical Report RAL-TR-2005-005, Rutherford Appleton Laboratory (2005, to appear) Google Scholar
  23. 23.
    Gould, N.I.M., Orban, D., Sartenaer, A., Toint, P.L.: Superlinear convergence of primal–dual interior point algorithms for nonlinear programming. SIAM J. Optim. 11, 974–1002 (2001) MATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Gupta, A., Ying, L.: On algorithms for finding maximum matchings in bipartite graphs. Technical Report RC 21576 (97320), IBM T.J. Watson Research Center, Yorktown Heights (25 October 1999) Google Scholar
  25. 25.
    Hagemann, M., Schenk, O.: Weighted matchings for preconditioning symmetric indefinite linear systems. SIAM J. Sci. Comput. 28, 403–420 (2006) MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20, 359–392 (1998) CrossRefMathSciNetGoogle Scholar
  27. 27.
    Liegmann, A.: Efficient solution of large sparse linear systems. Ph.D. thesis, ETH Zürich (1995) Google Scholar
  28. 28.
    Maurer, H., Mittelmann, H.D.: Optimization techniques for solving elliptic control problems with control and state constraints. part 1: Boundary control. Comput. Optim. Appl. 16, 29–55 (2000) MATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    Maurer, H., Mittelmann, H.D.: Optimization techniques for solving elliptic control problems with control and state constraints. part 2: Distributed control. Comput. Optim. Appl. 18, 141–160 (2001) MATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Mittelmann, H.D.: AMPL models. See
  31. 31.
    Mittelmann, H.D.: Sufficient optimality for discretized parabolic and elliptic control problems. In: Hoffmann, K.-H., Hoppe, R., Schulz, V. (eds.) Fast Solution of Discretized Optimization Problems. Birkhäuser, Basel (2001) Google Scholar
  32. 32.
    Neumaier, A.: Scaling and structural condition numbers. Linear Algebra Appl. 263, 157–165 (1997) MATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    Ng, E., Peyton, B.: Block sparse Cholesky algorithms on advanced uniprocessor computers. SIAM J. Sci. Comput. 14, 1034–1056 (1993) MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Nocedal, J., Wächter, A., Waltz, R.A.: Adaptive barrier strategies for nonlinear interior methods. Technical Report RC 23563, IBM T.J. Watson Research Center, Yorktown Heights, USA (March 2005) Google Scholar
  35. 35.
    Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (1999) MATHCrossRefGoogle Scholar
  36. 36.
    Olschowka, M., Neumaier, A.: A new pivoting strategy for Gaussian elimination. Linear Algebra Appl. 240, 131–151 (1996) CrossRefMathSciNetGoogle Scholar
  37. 37.
    Schenk, O., Gärtner, K.: Two-level scheduling in PARDISO: Improved scalability on shared memory multiprocessing systems. Parallel Comput. 28, 400–441 (2002) CrossRefGoogle Scholar
  38. 38.
    Schenk, O., Gärtner, K.: On fast factorization pivoting methods for symmetric indefinite systems. Electr. Trans. Numer. Anal. 23, 158–179 (2006) MATHGoogle Scholar
  39. 39.
    Schenk, O., Gärtner, K., Fichtner, W.: Efficient sparse LU factorization with left–right looking strategy on shared memory multiprocessors. BIT 40, 158–176 (2000) MATHCrossRefMathSciNetGoogle Scholar
  40. 40.
    Ulbrich, M., Ulbrich, S., Vicente, L.N.: A globally convergent primal–dual interior-point filter method for nonlinear programming. Math. Program. 100, 379–410 (2004) MATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    Wächter, A., Biegler, L.T.: On the implementation of a primal–dual interior-point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006) MATHCrossRefMathSciNetGoogle Scholar
  42. 42.
    Wächter, A., Biegler, L.T.: Line search filter methods for nonlinear programming: Motivation and global convergence. SIAM J. Optim. 16, 1–31 (2005) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Olaf Schenk
    • 1
  • Andreas Wächter
    • 2
  • Michael Hagemann
    • 1
  1. 1.Departement of Computer ScienceUniversity of BaselBaselSwitzerland
  2. 2.IBM T.J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations