Abstract
This paper studies a primal–dual interior/exterior-point path-following approach for linear programming that is motivated on using an iterative solver rather than a direct solver for the search direction. We begin with the usual perturbed primal–dual optimality equations. Under nondegeneracy assumptions, this nonlinear system is well-posed, i.e. it has a nonsingular Jacobian at optimality and is not necessarily ill-conditioned as the iterates approach optimality. Assuming that a basis matrix (easily factorizable and well-conditioned) can be found, we apply a simple preprocessing step to eliminate both the primal and dual feasibility equations. This results in a single bilinear equation that maintains the well-posedness property. Sparsity is maintained. We then apply either a direct solution method or an iterative solver (within an inexact Newton framework) to solve this equation. Since the linearization is well posed, we use affine scaling and do not maintain nonnegativity once we are close enough to the optimum, i.e. we apply a change to a pure Newton step technique. In addition, we correctly identify some of the primal and dual variables that converge to 0 and delete them (purify step).
We test our method with random nondegenerate problems and problems from the Netlib set, and we compare it with the standard Normal Equations NEQ approach. We use a heuristic to find the basis matrix. We show that our method is efficient for large, well-conditioned problems. It is slower than NEQ on ill-conditioned problems, but it yields higher accuracy solutions.
Similar content being viewed by others
References
Andersen, E.D., Ye, Y.: Combining interior-point and pivoting algorithms for linear programming. Manag. Sci. 42, 1719–1731 (1996)
Anstreichner, K.M.: Linear programming in O((n 3/ln n)L) operations. SIAM J. Optim. 9(4), 803–812 (1999) (electronic). Dedicated to John E. Dennis, Jr., on his 60th birthday
Ben-Tal, A., Nemirovski, A.S.: Robust convex optimization. Math. Oper. Res. 23(4), 769–805 (1998)
Ben-Tal, A., Nemirovski, A.S.: Robust solutions of uncertain linear programs. Oper. Res. Lett. 25(1), 1–13 (1999)
Ben-Tal, A., Nemirovski, A.: Robust solutions of linear programming problems contaminated with uncertain data. Math. Program. Ser. A 88(3), 411–424 (2000)
Benzi, M., Meyer, C.D., Tu̇ma, M.: A sparse approximate inverse preconditioner for the conjugate gradient method. SIAM J. Sci. Comput. 17(5), 1135–1149 (1996)
Benzi, M., Tu̇ma, M.: A sparse approximate inverse preconditioner for nonsymmetric linear systems. SIAM J. Sci. Comput. 19(3), 968–994 (1998)
Bergamaschi, L., Gondzio, J., Zilli, G.: Preconditioning indefinite systems in interior point methods for optimization. Comput. Optim. Appl. 28(2), 149–171 (2004)
Björck, Å.: Methods for sparse least squares problems. In: Bunch, J.R., Rose, D.J. (eds.) Sparse Matrix Computations, pp. 177–199. Academic Press, New York (1976)
Burke, J.: On the identification of active constraints II. The nonconvex case. SIAM J. Numer. Anal. 27(4), 1081–1103 (1990)
Burke, J.V., Moré, J.J.: On the identification of active constraints. SIAM J. Numer. Anal. 25(5), 1197–1211 (1988)
Burke, J.V., Moré, J.J.: Exposing constraints. SIAM J. Optim. 4(3), 573–595 (1994)
Chai, J.-S., Toh, K.-C.: Preconditioning and iterative solution of symmetric indefinite linear systems arising from interior point methods for linear programming. Comput. Optim. Appl. 36(2/3), 221–247 (2007)
De Leone, R., Mangasarian, O.L.: Serial and parallel solution of large scale linear programs by augmented Lagrangian successive overrelaxation. In: Optimization, Parallel Processing and Applications, Oberwolfach, Karlsruhe, 1987. Lecture Notes in Economics and Mathematical Systems, vol. 304, pp. 103–124. Springer, Berlin (1988)
Dennis, J.E. Jr., Schnabel, B.R.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Classics in Applied Mathematics, vol. 16. SIAM, Philadelphia (1996) (Corrected reprint of the 1983 original)
Dennis, J.E. Jr., Wolkowicz, H.: Sizing and least-change secant methods. SIAM J. Numer. Anal. 30(5), 1291–1314 (1993)
Dominguez, J., Gonzalez-Lima, M.D.: A primal–dual interior-point algorithm for quadratic programming. Numer. Algorithms 105, 1–30 (2006)
El-Bakry, A.S., Tapia, R.A., Zhang, Y.: A study of indicators for identifying zero variables in interior-point methods. SIAM Rev. 36(1), 45–72 (1994)
Fiacco, A.V., McCormick, G.P.: Nonlinear Programming Sequential Unconstrained Minimization Techniques. SIAM, Philadelphia (1990)
Freund, R.W., Gutknecht, M.H., Nachtigal, N.M.: An implementation of the look-ahead Lanczos algorithm for non-Hermitian matrices. SIAM J. Sci. Comput. 14, 137–158 (1993)
Freund, R.W., Jarre, F.: A QMR-based interior-point algorithm for solving linear programs. Math. Program. Ser. B 76, 183–210 (1996)
Golub, G.H., Pereyra, V.: The differentiation of pseudoinverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10, 413–432 (1973)
Gould, N.I.M., Orban, D., Sartenaer, A., Toint, Ph.L.: Componentwise fast convergence in the solution of full-rank systems of nonlinear equations. Tr/pa/00/56, CERFACS, Toulouse, France (2001)
Greenbaum, A.: Iterative Methods for Solving Linear Systems. SIAM, Philadelphia (1997)
Güler, O., Den Hertog, D., Roos, C., Terlaky, T., Tsuchiya, T.: Degeneracy in interior point methods for linear programming: a survey (Degeneracy in Optimization Problems). Ann. Oper. Res. 46/47(1–4), 107–138 (1993)
Hager, W.W.: The dual active set algorithm and the iterative solution of linear programs. In: Novel Approaches to Hard Discrete Optimization, Waterloo, ON, 2001. Fields Institute Communications, vol. 37, pp. 97–109. Am. Math. Soc., Providence (2003)
Júdice, J.J., Patricio, J., Portugal, L.F., Resende, M.G.C., Veiga, G.: A study of preconditioners for network interior point methods. Comput. Optim. Appl. 24(1), 5–35 (2003)
Kantorovich, L.V.: Functional analysis and applied mathematics. Uspekhi Mat. Nauk 3, 89–185 (1948) (Transl. by C. Benster as N.B.S. Rept. 1509, Washington, 1952)
Keil, C., Jansson, C.: Computational experience with rigorous error bounds for the Netlib linear programming library. Reliab. Comput. 12(4), 303–321 (2006)
Lu, S., Barlow, J.L.: Multifrontal computation with the orthogonal factors of sparse matrices. SIAM J. Matrix Anal. Appl. 17(3), 658–679 (1996)
Mangasarian, O.L.: Iterative solution of linear programs. SIAM J. Numer. Anal. 18(4), 606–614 (1981)
Matstoms, P.: The multifrontal solution of sparse linear least squares problems. Licentiat thesis, Department of Mathematics, Linköping University, Sweden (1991)
Matstoms, P.: Sparse QR factorization in MATLAB. ACM Trans. Math. Softw. 20, 136–159 (1994)
Mehrotra, S.: Implementations of affine scaling methods: approximate solutions of systems of linear equations using preconditioned conjugate gradient methods. ORSA J. Comput. 4(2), 103–118 (1992)
Mehrotra, S., Wang, J.-S.: Conjugate gradient based implementation of interior point methods for network flow problems. In: Linear and Nonlinear Conjugate Gradient-Related Methods, Seattle, WA, 1995, pp. 124–142. SIAM, Philadelphia (1996)
Mehrotra, S., Ye, Y.: Finding an interior point in the optimal face of linear programs. Math. Program. A 62(3), 497–515 (1993)
Oliveira, A.R.L., Sorensen, D.C.: A new class of preconditioners for large-scale linear systems from interior point methods for linear programming. Linear Algebra Appl. 394, 1–24 (2005)
Ordóñez, F., Freund, R.M.: Computational experience and the explanatory value of condition measures for linear optimization. SIAM J. Optim. 14(2), 307–333 (2003) (electronic)
Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)
Pang, J.: Error bounds in mathematical programming (Lectures on Mathematical Programming (ISMP97), Lausanne, 1997). Math. Program. Ser. B 79(1–3), 299–332 (1997)
Perez-Garzia, S.: Alternative iterative primal–dual interior-point algorithms for linear programming. Master’s thesis, Simon Bolivar University, Center for Statistics and Mathematical Software (CESMa), Venezuela (2003)
Perez-Garcia, S., Gonzalez-Lima, M.: On a non-inverse approach for solving the linear systems arising in primal–dual interior point methods for linear programming. Technical report 2004-01, Simon Bolivar University, Center for Statistical and Mathematical Software, Caracas, Venezuela (2004)
Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Winston/Wiley, Washington (1977) (Transl. ed. Fritz John)
Van der Sluis, A.: Condition numbers and equilibration of matrices. Numer. Math. 14, 14–23, (1969/1970)
Vanderbei, R.J.: Linear Programming: Foundations and Extensions. Kluwer Academic, Dordrecht (1998)
Vanderbei, R.J.: LOQO: an interior point code for quadratic programming (Interior Point Methods). Optim. Methods Softw. 11/12(1–4), 451–484 (1999)
Wei, H.: Numerical stability in linear programming and semidefinite programming. Ph.D. thesis, University of Waterloo (2006)
Wolkowicz, H.: Solving semidefinite programs using preconditioned conjugate gradients. Optim. Methods Softw. 19(6), 653–672 (2004)
Wright, M.H.: Ill-conditioning and computational error in interior methods for nonlinear programming. SIAM J. Optim. 9(1), 84–111 (1999) (electronic)
Wright, S.: Primal–Dual Interior-Point Methods. SIAM, Philadelphia (1996)
Wright, S.: Modifying SQP for degenerate problems. Technical report, Argonne National Laboratory (1997)
Wright, S.J.: Stability of linear equations solvers in interior-point methods. SIAM J. Matrix Anal. Appl. 16(4), 1287–1307 (1995)
Wright, S.J.: Stability of augmented system factorizations in interior-point methods. SIAM J. Matrix Anal. Appl. 18(1), 191–222 (1997)
Zhang, Y.: User’s guide to LIPSOL: linear-programming interior point solvers V0.4 (Interior Point Methods). Optim. Methods Softw. 11/12(1–4), 385–396 (1999)
Author information
Authors and Affiliations
Corresponding author
Additional information
M. Gonzalez-Lima research supported by Universidad Simón Bolívar (DID-GID001) and Conicit (project G97000592), Venezuela. H. Wei research supported by The Natural Sciences and Engineering Research Council of Canada and Bell Canada. H. Wolkowicz research supported by The Natural Sciences and Engineering Research Council of Canada.
URL: http://orion.math.uwaterloo.ca/~hwolkowi/henry/reports/ABSTRACTS.html. This report is a revision of the earlier CORR 2001-66.
Rights and permissions
About this article
Cite this article
Gonzalez-Lima, M., Wei, H. & Wolkowicz, H. A stable primal–dual approach for linear programming under nondegeneracy assumptions. Comput Optim Appl 44, 213–247 (2009). https://doi.org/10.1007/s10589-007-9157-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-007-9157-2