Computational Management Science

, Volume 7, Issue 2, pp 139–170 | Cite as

Towards a practical parallelisation of the simplex method

Original Paper


The simplex method is frequently the most efficient method of solving linear programming (LP) problems. This paper reviews previous attempts to parallelise the simplex method in relation to efficient serial simplex techniques and the nature of practical LP problems. For the major challenge of solving general large sparse LP problems, there has been no parallelisation of the simplex method that offers significantly improved performance over a good serial implementation. However, there has been some success in developing parallel solvers for LPs that are dense or have particular structural properties. As an outcome of the review, this paper identifies scope for future work towards the goal of developing parallel implementations of the simplex method that are of practical value.


Linear programming Simplex method Sparse Parallel computing 

Mathematics Subject Classification (2000)



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Agrawal A, Blelloch GE, Krawitz RL, Phillips CA (1989) Four vector–matrix primitives. In: ACM symposium on parallel algorithms and architectures, pp 292–302Google Scholar
  2. Amdahl GM (1967) Validity of the single-processor approach to achieving large scale computing capabilities. In: AFIPS conference proceedings, vol 30. AFIPS Press, Reston, pp 483–485Google Scholar
  3. Amestoy PR, Duff IS, L’Excellent J-Y (2000) Multifrontal parallel distributed symmetric and unsymmetric solvers. Comput Methods Appl Mech Eng 184: 501–520CrossRefGoogle Scholar
  4. Amestoy PR, Duff IS, Koster J, L’Excellent J-Y (2001) A fully asynchronous multifrontal solver using distributed dynamic scheduling. SIAM J Matrix Anal Appl 23(1): 15–41CrossRefGoogle Scholar
  5. Amestoy PR, Guermouche A, L’Excellent J-Y, Pralet S (2006) Hybrid scheduling for the parallel solution of linear systems. Parallel Comput 32(2): 136–156CrossRefGoogle Scholar
  6. Amestoy P, Li XS, Ng EG (2007a) Diagonal Markowitz scheme with local symmetrization. SIAM J Matrix Anal Appl 29(1): 228–244CrossRefGoogle Scholar
  7. Amestoy P, Pralet S, Li XS (2007b) Unsymmetric orderings using a constrained Markowitz scheme. SIAM J Matrix Anal Appl 29(1): 302–327CrossRefGoogle Scholar
  8. Aykanat C, Pinar A, Çatalyürek ÜV (2004) Permuting sparse rectangular matrices into block-diagonal form. SIAM J Sci Comput 25(6): 1860–1879CrossRefGoogle Scholar
  9. Babaev DA, Mardanov SS (1991) A parallel algorithm for solving linear programming problems. Zh Vychislitel’noi Matematiki Matematicheskoi Fiziki 31(1): 86–95Google Scholar
  10. Badr E-S, Moussa M, Papparrizos K, Samaras N, Sifaleras A (2006) Some computational results on MPI parallel implementations of dense simplex method. Trans Eng Comput Technol 17: 228–231Google Scholar
  11. Barr RS, Hickman BL (1994) Parallel simplex for large pure network problems: computational testing and sources of speedup. Oper Res 42(1): 65–80CrossRefGoogle Scholar
  12. Beasley JE (1990) Linear programming on Cray supercomputers. J Oper Res Soc 41(2): 133–139CrossRefGoogle Scholar
  13. Benders JF (1962) Partitioning procedures for solving mixed-variables programming problems. Numer Math 4: 238–252CrossRefGoogle Scholar
  14. Bixby RE (2002) Solving real-world linear programs: a decade and more of progress. Oper Res 50(1): 3–15CrossRefGoogle Scholar
  15. Bixby RE, Martin A (2000) Parallelizing the dual simplex method. INFORMS J Comput 12: 45–56CrossRefGoogle Scholar
  16. Bixby RE, Fenelon M, Gu Z, Rothberg E, Wunderling R (2000) MIP: theory and practice closing the gap. In: Powell MJD, Scholtes S (eds) System modelling and optimization: methods, theory and applications. Kluwer, The Netherlands, pp 19–49Google Scholar
  17. Bodurog̃lu İİ (1997) Scalable massively parallel simplex algorithms for block-structured linear programs. Ph.D. thesis, GSAS, Columbia University, New YorkGoogle Scholar
  18. Boffey TB, Hay R (1989) Implementing parallel simplex algorithms. In: CONPAR 88. Cambridge, UK, Cambridge University Press, pp 169–176Google Scholar
  19. Carolan WJ, Hill JE, Kennington JL, Niemi S, Wichmann SJ (1990) An empirical evaluation of the KORBX algorithms for military airlift applications. Oper Res 38(2): 240–248CrossRefGoogle Scholar
  20. Chang MD, Engquist M, Finkel R, Meyer RR (1988) A parallel algorithm for generalized networks. Ann Oper Res 14: 125–145CrossRefGoogle Scholar
  21. Crowder H, Hattingh JM (1975) Partially normalized pivot selection in linear programming. Math Program Study 4: 12–25Google Scholar
  22. Cvetanovic Z, Freedman EG, Nofsinger C (1991) Efficient decomposition and performance of parallel PDE, FFT, Monte-Carlo simulations, simplex, and sparse solvers. J Supercomput 5: 19–38CrossRefGoogle Scholar
  23. Dantzig GB (1960) The decomposition principle for linear programs. Oper Res 8: 101–111CrossRefGoogle Scholar
  24. Dantzig GB, Orchard-Hays W (1954) The product form for the inverse in the simplex method. Math Comput 8: 64–67CrossRefGoogle Scholar
  25. Demmel JW, Eisenstat SC, Gilbert JR, Li XS, Liu JWH (1999) A supernodal approach to sparse partial pivoting. SIAM J Matrix Anal Appl 20(3): 720–755CrossRefGoogle Scholar
  26. Dempster MAH, Thompson RT (1998) Parallelization and aggregation of nested Benders decomposition. Ann Oper Res 81: 163–187CrossRefGoogle Scholar
  27. Eckstein J, Bodurog̃lu İİ, Polymenakos L, Goldfarb D (1995) Data-parallel implementations of dense simplex methods on the connection machine CM-2. ORSA J Comput 7(4): 402–416Google Scholar
  28. Evans DJ, Hatzopoulos M (1979) A parallel linear system solver. Int J Comput Math 7: 227–238CrossRefGoogle Scholar
  29. Ferris MC, Horn JD (1998) Partitioning mathematical programs for parallel solution. Math Program 80: 35–61Google Scholar
  30. Finkel RA (1987) Large-grain parallelism: three case studies. In: Jamieson LH, Gannon D, Douglas RJ (eds) The characteristics of parallel algorithms. MIT Press, Cambridge, pp 21–63Google Scholar
  31. Forrest JJ, Goldfarb D (1992) Steepest-edge simplex algorithms for linear programming. Math Program 57: 341–374CrossRefGoogle Scholar
  32. Forrest JJH, Tomlin JA (1990) Vector processing in the simplex and interior methods for linear programming. Ann Oper Res 22: 71–100CrossRefGoogle Scholar
  33. Gay DM (1985) Electronic mail distribution of linear programming test problems. Math Program Soc COAL Newsl 13: 10–12Google Scholar
  34. Gill PE, Murray W, Saunders MA, Wright MH (1989) A practical anti-cycling procedure for linearly constrained optimization. Math Program 45: 437–474CrossRefGoogle Scholar
  35. Gnanendran SK, Ho JK (1993) Load balancing in the parallel optimization of block-angular linear programs. Math Program 62: 41–67CrossRefGoogle Scholar
  36. Goldfarb D, Reid JK (1977) A practical steepest-edge simplex algorithm. Math Program 12: 361–371CrossRefGoogle Scholar
  37. Hall JAJ (2005) SYNPLEX: a task-parallel scheme for the revised simplex method. In: Contributed talk at the second international workshop on combinatorial scientific computing (CSC05), June 2005Google Scholar
  38. Hall JAJ (2007) Parallel basis matrix triangularisation for hyper-sparse LP problems. In: Contributed talk the IMA conference on numerical linear algebra and optimisation, September 2007Google Scholar
  39. Hall JAJ, McKinnon KIM (1992) Update procedures for the parallel revised simplex method. Technical report MSR 92-13, Department of Mathematics and Statistics, University of EdinburghGoogle Scholar
  40. Hall JAJ, McKinnon KIM (1996) PARSMI: a parallel revised simplex algorithm incorporating minor iterations and Devex pricing. In: Waśniewski J, Dongarra J, Madsen K, Olesen D (eds) Applied parallel computing, vol 1184. Lecture notes in computer science. Springer, Berlin, pp 67–76Google Scholar
  41. Hall JAJ, McKinnon KIM (1998) ASYNPLEX: an asynchronous parallel revised simplex method algorithm. Ann Oper Res 81: 27–49CrossRefGoogle Scholar
  42. Hall JAJ, McKinnon KIM (1999) Exploiting hyper-sparsity in the revised simplex method. Technical report MS99-014. Department of Mathematics and Statistics, University of EdinburghGoogle Scholar
  43. Hall JAJ, McKinnon KIM (2005) Hyper-sparsity in the revised simplex method and how to exploit it. Comput Optim Appl 32(3): 259–283CrossRefGoogle Scholar
  44. Harris PMJ (1973) Pivot selection methods of the Devex LP code. Math Program 5: 1–28CrossRefGoogle Scholar
  45. Helgason RV, Kennington LJ, Zaki HA (1988) A parallelisation of the simplex method. Ann Oper Res 14: 17–40CrossRefGoogle Scholar
  46. Hiller RS, Eckstein J (1993) Stochastic dedication: designing fixed income portfolios using massively parallel Benders decomposition. Manage Sci 39(11): 1422–1438CrossRefGoogle Scholar
  47. Ho JK, Lee TC, Sundarraj RP (1988) Decomposition of linear programs using parallel computation. Math Program 42: 391–405CrossRefGoogle Scholar
  48. Ho JK, Sundarraj RP (1994) On the efficacy of distributed simplex algorithms for linear programming. Comput Optim Appl 3(4): 349–363CrossRefGoogle Scholar
  49. Kaul RN (1965) An extension of generalized upper bounding techniques for linear programming. Technical Report ORC 65-27. In: Center OR, Berkley UC, San Fransisco, CAGoogle Scholar
  50. Klabjan D, Johnson EL, Nemhauser GL (2000) A parallel primal-dual simplex algorithm. Oper Res Lett 27: 47–55CrossRefGoogle Scholar
  51. Kumar V, Grama A, Gupta A, Karypis G (2003) Introduction to parallel computing: design and analysis of algorithms, 2nd edn. Addison–Wesley, New YorkGoogle Scholar
  52. Lentini M, Reinoza A, Teruel A, Guillen A (1995) SIMPAR: a parallel sparse simplex. Comput Appl Math 14(1): 49–58Google Scholar
  53. Li XS, Demmel JW (2003) SuperLU_DIST: a scalable distributed-memory sparse direct solver for unsymmetric linear systems. ACM Trans Math Softw 29(2): 110–140CrossRefGoogle Scholar
  54. Luo J, Reijns GL (1992) Linear programming on transputers. In: van Leeuwen J (eds) Algorithms, software, architecture IFIP. transactions A (computer science and technology). Elsevier, Amsterdam, pp 525–534Google Scholar
  55. Luo J, Reijns GL, Bruggeman F, Lindfield GR (1991) A survey of parallel algorithms for linear programming. In: Deprettere EF, van der Veen A-J (eds) Algorithms and parallel VLSI architectures, vol B. Elsevier, Amsterdam, pp 485–490Google Scholar
  56. Markowitz H (1957) The elimination form of the inverse and its application to linear programming. Manage Sci 3: 255–296CrossRefGoogle Scholar
  57. Maros I, Mitra G (2000) Investigating the sparse simplex algorithm on a distributed memory multiprocessor. Parallel Comput 26: 151–170CrossRefGoogle Scholar
  58. McKinnon KIM, Plab F (1997a) A modified Markowitz criterion to increase parallelism in inverse factors of sparse matrices. Technical report, Department of Mathematics and Statistics, University of EdinburghGoogle Scholar
  59. McKinnon KIM, Plab F (1997b) An upper bound on parallelism in the forward transformation within the revised simplex method. Technical report, Department of Mathematics and Statistics, University of EdinburghGoogle Scholar
  60. Medhi D (1990) Parallel bundle-based decomposition algorithm for large-scale structured mathematical programming problems. Ann Oper Res 22: 101–127CrossRefGoogle Scholar
  61. Mittelmann HD (2008) Benchmarks for optimization software. Accessed February 2008
  62. Nielsen SN, Zenios SA (1997) Scalable Benders decomposition for stochastic linear programming. Parallel Comput 23: 1069–1088CrossRefGoogle Scholar
  63. Orchard-Hays W (1968) Advanced linear programming computing techniques. McGraw-Hill, New YorkGoogle Scholar
  64. Peters J (1990) The network simplex method on a multiprocessor. Networks 20: 845–859CrossRefGoogle Scholar
  65. Pfefferkorn CE, Tomlin JA (1976) Design of a linear programming system for the ILLIAC IV. Technical report SOL 76-8. Systems Optimization Laboratory, Stanford UniversityGoogle Scholar
  66. Pinar A, Aykanat C (1996) An effective model to decompose linear programs for parallel solution. In: Waśniewski J, Dongarra J, Madsen K, Olesen D (eds) Applied parallel computing. Lecture notes in computer science, vol 1184. Springer, Heidelberg, pp 592–601Google Scholar
  67. Rosen JB, Maier RS (1990) Parallel solution of large-scale, block-angular linear programs. Ann Oper Res 22: 23–41CrossRefGoogle Scholar
  68. Shu W (1995) Parallel implementation of a sparse simplex algorithm on MIMD distributed memory computers. J Parallel Distrib Comput 31(1): 25–40CrossRefGoogle Scholar
  69. Stunkel CB (1988) Linear optimization via message-based parallel processing. In: International conference on parallel processing, vol III, August 1988, pp 264–271Google Scholar
  70. Suhl UH, Suhl LM (1990) Computing sparse LU factorizations for large-scale linear programming bases. ORSA J Comput 2(4): 325–335Google Scholar
  71. Thomadakis ME, Liu J-C (1996) An efficient steepest-edge simplex algorithm for SIMD computers. In: International conference on supercomputing, pp 286–293Google Scholar
  72. Tomlin JA (1972) Pivoting for size and sparsity in linear programming inversion routines. J Inst Maths Applics 10: 289–295CrossRefGoogle Scholar
  73. Wunderling R (1996a) Parallelizing the simplex algorithm. ILAY workshop on linear algebra in optimzation, AlbiGoogle Scholar
  74. Wunderling R (1996b) Paralleler und objektorientierter simplex. Technical report TR-96-09, Konrad-Zuse-Zentrum für Informationstechnik BerlinGoogle Scholar
  75. Yarmish G (2001) A distributed implementation of the simplex method. Ph.D. thesis, Polytechnic University, Brooklyn, NY, March 2001Google Scholar
  76. Zenios SA (1989) Parallel numerical optimization: current status and annotated bibliography. ORSA J Comput 1(1)(Winter): 20–43Google Scholar

Copyright information

© Springer-Verlag 2008

Authors and Affiliations

  1. 1.School of MathematicsUniversity of EdinburghEdinburghUK

Personalised recommendations