A preconditioning technique for Schur complement systems arising in stochastic optimization
 Cosmin G. Petra,
 Mihai Anitescu
 … show all 2 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
Deterministic sample average approximations of stochastic programming problems with recourse are suitable for a scenariobased parallelization. In this paper the parallelization is obtained by using an interiorpoint method and a Schur complement mechanism for the interiorpoint linear systems. However, the direct linear solves involving the dense Schur complement matrix are expensive, and adversely affect the scalability of this approach. We address this issue by proposing a stochastic preconditioner for the Schur complement matrix and by using Krylov iterative methods for the solution of the dense linear systems. The stochastic preconditioner is built based on a subset of existing scenarios and can be assembled and factorized on a separate process before the computation of the Schur complement matrix finishes on the remaining processes. The expensive factorization of the Schur complement is removed from the parallel execution flow and the scaling of the optimization solver is considerably improved with this approach. The spectral analysis indicates an exponentially fast convergence in probability to 1 of the eigenvalues of the preconditioned matrix with the number of scenarios incorporated in the preconditioner. Numerical experiments performed on the relaxation of a unit commitment problem show good performance, in terms of both the accuracy of the solution and the execution time.
 Altman, A., Gondzio, J. (1999) Regularized symmetric indefinite systems in interiorpoint methods for linear and quadratic optimization. Optim. Methods Softw. 11: pp. 275302 CrossRef
 Benzi, M., Golub, G.H., Liesen, J. (2005) Numerical solution of saddle point problems. Acta Numer. 14: pp. 1137 CrossRef
 Bergamaschi, L., Gondzio, J., Zilli, G. (2004) Preconditioning indefinite systems in interior point methods for optimization. Comput. Optim. Appl. 28: pp. 149171 CrossRef
 Birge, J.R.: Current trends in stochastic programming computation and applications. Tech. rep., Department of Industrial and Operations Engineering, University of Michigan, Ann Harbour, Michigan (1995)
 Birge, J.R., Holmes, D.F. (1992) Efficient solution of two stage stochastic linear programs using interior point methods. Comput. Optim. Appl. 1: pp. 245276 CrossRef
 Birge, J.R., Louveaux, F. (1997) Introduction to Stochastic Programming. Springer, New York
 Birge, J.R., Qi, L. (1988) Computing blockangular Karmarkar projections with applications to stochastic programming. Manag. Sci. 34: pp. 14721479 CrossRef
 Birge, J.R., Chen, X., Qi, L.: A stochastic Newton method for stochastic quadratic programs with recourse. Tech. rep., Applied Mathematics Preprint AM94/9, School of Mathematics, The University of New South Wales (1995)
 Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C. (1997) ScaLAPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia CrossRef
 Bunch, J.R., Kaufman, L. (1977) Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput. 31: pp. 163179 CrossRef
 Bunch, J.R., Parlett, B.N. (1971) Direct methods for solving symmetric indefinite systems of linear equations. SIAM J. Numer. Anal. 8: pp. 639655 CrossRef
 Cafieri, S., D’Apuzzo, M., Marino, M., Mucherino, A., Toraldo, G. (2006) Interiorpoint solver for largescale quadratic programming problems with bound constraints. J. Optim. Theory Appl. 129: pp. 5575 CrossRef
 Castro, J. (2000) A specialized interiorpoint algorithm for multicommodity network flows. SIAM J. Optim. 10: pp. 852877 CrossRef
 Constantinescu, E.M., Zavala, V.M., Rocklin, M., Lee, S., Anitescu, M. (2011) A computational framework for uncertainty quantification and stochastic optimization in unit commitment with wind power generation. IEEE Trans. Power Syst. 26: pp. 431441 CrossRef
 Czyzyk, J., Mehrotra, S., Wright, S.J.: PCx user guide. Tech. Rep. OTC 96/01, Optimization Technology Center, Argonne National Laboratory and Northwestern University (1996)
 Dantzig, G.B., Infanger, G. (1992) Largescale stochastic linear programs—Importance sampling and Benders decomposition. Computational and Applied Mathematics, I. NorthHolland, Amsterdam, pp. 111120
 D’Apuzzo, M., Simone, V., Serafino, D. (2010) On mutual impact of numerical linear algebra and largescale optimization with focus on interior point methods. Comput. Optim. Appl. 45: pp. 283310 CrossRef
 Dollar, H.S., Wathen, A.J. (2006) Approximate factorization constraint preconditioners for saddlepoint matrices. SIAM J. Sci. Comput. 27: pp. 15551572 CrossRef
 Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J. (2006) Implicitfactorization preconditioning and iterative solvers for regularized saddlepoint systems. SIAM J. Matrix Anal. Appl. 28: pp. 170189 CrossRef
 Ermoliev, Y.M. (1988) Stochastic quasigradient methods. Numerical Techniques for Stochastic Optimization. Springer, Berlin, pp. 141185 CrossRef
 Gertz, E.M., Wright, S.J. (2003) Objectoriented software for quadratic programming. ACM Trans. Math. Softw. 29: pp. 5881 CrossRef
 Golub, G.H., Loan, C.F. (1996) Matrix Computations. Johns Hopkins University Press, Baltimore
 Gondzio, J. (1995) HOPDM (version 2.12)—a fast LP solver based on a primaldual interior point method. Eur. J. Oper. Res. 85: pp. 221225 CrossRef
 Gondzio, J., Grothey, A. (2005) Direct solution of linear systems of size 109 arising in optimization with interior point methods. PPAM. pp. 513525
 Gondzio, J., Grothey, A. (2007) Parallel interiorpoint solver for structured quadratic programs: application to financial planning problems. Ann. Oper. Res. 152: pp. 319339 CrossRef
 Gondzio, J., Grothey, A. (2009) Exploiting structure in parallel implementation of interior point methods for optimization. Comput. Manag. Sci. 6: pp. 135160 CrossRef
 Gondzio, J., Makowski, M. (1995) Solving a class of LP problems with a primal–dual logarithmic barrier method. Eur. J. Oper. Res. 80: pp. 184192 CrossRef
 Gondzio, J., Sarkissian, R. (2000) Parallel interior point solver for structured linear programs. Math. Program. 96: pp. 561584 CrossRef
 Gould, N.I.M., Hribar, M.E., Nocedal, J. (2001) On the solution of equality constrained quadratic programming problems arising in optimization. SIAM J. Sci. Comput. 23: pp. 13761395 CrossRef
 Greenbaum, A. (1997) Iterative Methods for Solving Linear Systems. SIAM Series on Frontiers in Applied Mathematics, Philadelphia CrossRef
 Güler, O. (1993) Existence of interior points and interior paths in nonlinear monotone complementarity problems. Math. Oper. Res. 18: pp. 128147 CrossRef
 Higle, J.L., Sen, S. (1991) Stochastic decomposition: an algorithm for twostage linear programs with recourse. Math. Oper. Res. 16: pp. 650669 CrossRef
 Keller, C., Gould, N.I.M., Wathen, A.J. (2000) Constraint preconditioning for indefinite linear systems. SIAM J. Matrix Anal. Appl. 21: pp. 13001317 CrossRef
 King, A.J. (1988) An implementation of the Lagrangian finitegeneration method. Numerical Techniques for Stochastic Optimization. Springer, Berlin, pp. 295311 CrossRef
 Linderoth, J., Wright, S.J. (2003) Decomposition algorithms for stochastic programming on a computational grid. Comput. Optim. Appl. 24: pp. 207250 CrossRef
 Luks̆an, L., Vlc̆ek, J. (1998) Indefinitely preconditioned inexact Newton method for large sparse equality constrained nonlinear programming problems. Numer. Linear Algebra Appl. 5: pp. 219247 CrossRef
 Lustig, I.J., Marsten, R.E., Shanno, D.F. (1992) On implementing Mehrotra’s predictor–corrector interiorpoint method for linear programming. SIAM J. Optim. 2: pp. 435449 CrossRef
 McKay, M.D., Beckman, R.J., Conover, W.J. (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21: pp. 239245
 Mehrotra, S. (1992) On the implementation of a primaldual interior point method. SIAM J. Optim. 2: pp. 575601 CrossRef
 Mehrotra, S., Ozevin, M.G. (2007) Decompositionbased interior point methods for twostage stochastic semidefinite programming. SIAM J. Optim. 18: pp. 206222 CrossRef
 Mehrotra, S., Ozevin, M.G. (2009) Decomposition based interior point methods for twostage stochastic convex quadratic programs with recourse. Oper. Res. 57: pp. 964974 CrossRef
 Mehrotra, S., Ozevin, M.G. (2009) On the implementation of interior point decomposition algorithms for twostage stochastic conic programs. SIAM J. Optim. 19: pp. 18461880 CrossRef
 Monteiro, R.D.C., Pang, J.S. (1996) Properties of an interiorpoint mapping for mixed complementarity problems. Math. Oper. Res. 21: pp. 629654 CrossRef
 Murphy, M.F., Golub, G.H., Wathen, A.J. (2000) A note on preconditioning for indefinite linear systems. SIAM J. Sci. Comput. 21: pp. 19691972 CrossRef
 Pflug, G.C., Halada, L. (2003) A note on the recursive and parallel structure of the Birge and Qi factorization for tree structured linear programs. Comput. Optim. Appl. 24: pp. 251265 CrossRef
 Poulson, J., Marker, B., van de Geijn, R.A.: Elemental: A new framework for distributed memory dense matrix computations (flame working note #44). Tech. Rep., Institute for Computational Engineering and Sciences. The University of Texas at Austin (2010)
 Rockafellar, R.T., Wets, R.J.B. (1986) A Lagrangian finite generation technique for solving linearquadratic problems in stochastic programming. Math. Program. Stud. 28: pp. 6393 CrossRef
 Rockafellar, R.T., Wets, R.J.B. (1991) Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16: pp. 119147 CrossRef
 Rosa, C.H., Ruszczyński, A. (1996) On augmented Lagrangian decomposition methods for multistage stochastic programs. Ann. Oper. Res. 64: pp. 289309 CrossRef
 Ruszczyński, A. (1986) A regularized decomposition method for minimizing a sum of polyhedral functions. Math. Program. 35: pp. 309333 CrossRef
 Shapiro, A., Dentcheva, D., Ruszczyński, A. (2009) Lectures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia CrossRef
 Slyke, R., Wets, R.J. (1969) Lshaped linear programs with applications to control and stochastic programming. SIAM J. Appl. Math. 17: pp. 638663 CrossRef
 Wright, S.J. (1997) PrimalDual InteriorPoint Methods. Society for Industrial and Applied Mathematics, Philadelphia CrossRef
 Zavala, V.M., Laird, C.D., Biegler, L.T. (2008) Interiorpoint decomposition approaches for parallel solution of largescale nonlinear parameter estimation problems. Chem. Eng. Sci. 63: pp. 48344845 CrossRef
 Zavala, V.M., Constantinescu, E.M., Krause, T., Anitescu, M. (2009) Online economic optimization of energy systems using weather forecast information. J. Process Control 19: pp. 17251736 CrossRef
 Zhang, D., Zhang, Y. (1996) A Mehrotratype predictorcorrector algorithm with polynomiality and Qsubquadratic convergence. Ann. Oper. Res. 62: pp. 131150 CrossRef
 Zhang, Y.: Solving largescale linear programs by interiorpoint methods under the Matlab environment. Tech. Rep. TR9601, University of Maryland Baltimore County (1996)
 Zhao, G. (2001) A logbarrier method with Benders decomposition for solving twostage stochastic linear programs. Math. Program. 90: pp. 507536 CrossRef
 Title
 A preconditioning technique for Schur complement systems arising in stochastic optimization
 Journal

Computational Optimization and Applications
Volume 52, Issue 2 , pp 315344
 Cover Date
 20120601
 DOI
 10.1007/s105890119418y
 Print ISSN
 09266003
 Online ISSN
 15732894
 Publisher
 Springer US
 Additional Links
 Topics
 Keywords

 Stochastic programming
 Saddlepoint preconditioning
 Krylov methods
 Interiorpoint method
 Sample average approximations
 Parallel computing
 Industry Sectors
 Authors

 Cosmin G. Petra ^{(1)}
 Mihai Anitescu ^{(1)}
 Author Affiliations

 1. Mathematics and Computer Science Division, Argonne National Laboratory, 9700 South Cass Avenue, Argonne, IL, 60439, USA