Computational Optimization and Applications

, Volume 55, Issue 3, pp 571–596 | Cite as

Parallel distributed-memory simplex for large-scale stochastic LP problems

  • Miles Lubin
  • J. A. Julian Hall
  • Cosmin G. Petra
  • Mihai Anitescu


We present a parallelization of the revised simplex method for large extensive forms of two-stage stochastic linear programming (LP) problems. These problems have been considered too large to solve with the simplex method; instead, decomposition approaches based on Benders decomposition or, more recently, interior-point methods are generally used. However, these approaches do not provide optimal basic solutions, which allow for efficient hot-starts (e.g., in a branch-and-bound context) and can provide important sensitivity information. Our approach exploits the dual block-angular structure of these problems inside the linear algebra of the revised simplex method in a manner suitable for high-performance distributed-memory clusters or supercomputers. While this paper focuses on stochastic LPs, the work is applicable to all problems with a dual block-angular structure. Our implementation is competitive in serial with highly efficient sparsity-exploiting simplex codes and achieves significant relative speed-ups when run in parallel. Additionally, very large problems with hundreds of millions of variables have been successfully solved to optimality. This is the largest-scale parallel sparsity-exploiting revised simplex implementation that has been developed to date and the first truly distributed solver. It is built on novel analysis of the linear algebra for dual block-angular LP problems when solved by using the revised simplex method and a novel parallel scheme for applying product-form updates.


Simplex method Parallel computing Stochastic optimization Block-angular 


  1. 1.
    Amdahl, G.M.: Validity of the single processor approach to achieving large scale computing capabilities. In: Proceedings of the April 18–20, 1967, Spring Joint Computer Conference, AFIPS ’67 (Spring), pp. 483–485. ACM, New York (1967) CrossRefGoogle Scholar
  2. 2.
    Aykanat, C., Pinar, A., Çatalyürek, U.V.: Permuting sparse rectangular matrices into block-diagonal form. SIAM J. Sci. Comput. 25, 1860–1879 (2004) MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Bennett, J.M.: An approach to some structured linear programming problems. Oper. Res. 14(4), 636–645 (1966) MATHCrossRefGoogle Scholar
  4. 4.
    Birge, J., Louveaux, F.: Introduction to Stochastic Programming, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer, New York (2011) MATHCrossRefGoogle Scholar
  5. 5.
    Bisschop, J., Meeraus, A.: Matrix augmentation and partitioning in the updating of the basis inverse. Math. Program. 13, 241–254 (1977) MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Dantzig, G.B., Orchard-Hays, W.: The product form for the inverse in the simplex method. Math. Tables Other Aids Comput. 8(46), 64–67 (1954) MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    Davis, T.: Direct Methods for Sparse Linear Systems. Fundamentals of Algorithms. Society for Industrial and Applied Mathematics, Philadelphia (2006) CrossRefGoogle Scholar
  8. 8.
    Duff, I.S., Scott, J.A.: A parallel direct solver for large sparse highly unsymmetric linear systems. ACM Trans. Math. Softw. 30, 95–117 (2004) MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Forrest, J.J., Goldfarb, D.: Steepest-edge simplex algorithms for linear programming. Math. Program. 57, 341–374 (1992) MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Forrest, J.J.H., Tomlin, J.A.: Updated triangular factors of the basis to maintain sparsity in the product form simplex method. Math. Program. 2, 263–278 (1972) MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Gilbert, J.R., Peierls, T.: Sparse partial pivoting in time proportional to arithmetic operations. SIAM J. Sci. Stat. Comput. 9, 862–874 (1988) MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: A practical anti-cycling procedure for linearly constrained optimization. Math. Program. 45, 437–474 (1989) MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996) MATHGoogle Scholar
  14. 14.
    Gondzio, J., Grothey, A.: Exploiting structure in parallel implementation of interior point methods for optimization. Comput. Manag. Sci. 6, 135–160 (2009) MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Grama, A., Karypis, G., Kumar, V., Gupta, A.: Introduction to Parallel Computing, 2nd edn. Addison-Wesley, Reading (2003) Google Scholar
  16. 16.
    Gropp, W., Thakur, R., Lusk, E.: Using MPI-2: Advanced Features of the Message Passing Interface, 2nd edn. MIT Press, Cambridge (1999) Google Scholar
  17. 17.
    Hall, J., McKinnon, K.: Hyper-sparsity in the revised simplex method and how to exploit it. Comput. Optim. Appl. 32, 259–283 (2005) MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Hall, J.A.J., Smith, E.: A high performance primal revised simplex solver for row-linked block angular linear programming problems. Tech. Rep. ERGO-12-003, School of Mathematics, University of Edinburgh (2012) Google Scholar
  19. 19.
    Harris, P.M.J.: Pivot selection methods of the DEVEX LP code. Math. Program. 5, 1–28 (1973) MATHCrossRefGoogle Scholar
  20. 20.
    Kall, P.: Computational methods for solving two-stage stochastic linear programming problems. Z. Angew. Math. Phys. 30, 261–271 (1979) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Koberstein, A.: The dual simplex method, techniques for a fast and stable implementation. Ph.D. thesis, Universität Paderborn, Paderborn, Germany (2005) Google Scholar
  22. 22.
    Koberstein, A.: Progress in the dual simplex algorithm for solving large scale LP problems: techniques for a fast and stable implementation. Comput. Optim. Appl. 41, 185–204 (2008) MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Lasdon, L.S.: Optimization Theory for Large Systems. Macmillan, New York (1970) MATHGoogle Scholar
  24. 24.
    Linderoth, J., Wright, S.: Decomposition algorithms for stochastic programming on a computational grid. Comput. Optim. Appl. 24, 207–250 (2003) MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Lubin, M., Petra, C.G., Anitescu, M., Zavala, V.: Scalable stochastic optimization of complex energy systems. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’11, pp. 64:1–64:64. ACM, New York (2011) Google Scholar
  26. 26.
    Markowitz, H.M.: The elimination form of the inverse and its application to linear programming. Manag. Sci. 3(3), 255–269 (1957) MathSciNetMATHCrossRefGoogle Scholar
  27. 27.
    Maros, I.: Computational Techniques of the Simplex Method. Kluwer Academic, Norwell (2003) MATHCrossRefGoogle Scholar
  28. 28.
    Stern, J., Vavasis, S.: Active set methods for problems in column block angular form. Comput. Appl. Math. 12, 199–226 (1993) MathSciNetMATHGoogle Scholar
  29. 29.
    Strazicky, B.: Some results concerning an algorithm for the discrete recourse problem. In: Stochastic Programming (Proc. Internat. Conf., Univ. Oxford, Oxford, 1974), pp. 263–274. Academic Press, London (1980) Google Scholar
  30. 30.
    Suhl, L.M., Suhl, U.H.: A fast LU update for linear programming. Ann. Oper. Res. 43, 33–47 (1993) MathSciNetMATHCrossRefGoogle Scholar
  31. 31.
    Suhl, U.H., Suhl, L.M.: Computing sparse LU factorizations for large-scale linear programming bases. ORSA J. Comput. 2(4), 325 (1990) MATHCrossRefGoogle Scholar
  32. 32.
    Vladimirou, H., Zenios, S.: Scalable parallel computations for large-scale stochastic programming. Ann. Oper. Res. 90, 87–129 (1999) MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York (outside the USA) 2013

Authors and Affiliations

  • Miles Lubin
    • 1
  • J. A. Julian Hall
    • 2
  • Cosmin G. Petra
    • 1
  • Mihai Anitescu
    • 1
  1. 1.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA
  2. 2.School of MathematicsUniversity of EdinburghEdinburghUK

Personalised recommendations