# A linear programming primer: from Fourier to Karmarkar

## Abstract

The story of linear programming is one with all the elements of a grand historical drama. The original idea of testing if a polyhedron is non-empty by using a variable elimination to project down one dimension at a time until a tautology emerges dates back to a paper by Fourier in 1823. This gets re-invented in the 1930s by Motzkin. The real interest in linear programming happens during World War II when mathematicians ponder best ways of utilising resources at a time when they are constrained. The problem of optimising a linear function over a set of linear inequalities becomes the focus of the effort. Dantzig’s Simplex Method is announced and the Rand Corporation becomes a hot bed of computational mathematics. The range of applications of this modelling approach grows and the powerful machinery of numerical analysis and numerical linear algebra becomes a major driver for the advancement of computing machines. In the 1970s, constructs of theoretical computer science indicate that linear programming may in fact define the frontier of tractable problems that can be solved effectively on large instances. This raised a series of questions and answers: Is the Simplex Method a polynomial-time method and if not can we construct novel polynomial time methods, etc. And that is how the Ellipsoid Method from the Soviet Union and the Interior Point Method from Bell Labs make their way into this story as the heroics of Khachiyan and Karmarkar. We have called this paper a primer on linear programming since it only gives the reader a quick narrative of the grand historical drama. Hopefully it motivates a young reader to delve deeper and add another chapter.

## Keywords

Linear inequalities Linear programming Optimisation Duality Simplex method Ellipsoid method Interior point method Monotone## Notes

### Acknowledgements

The authors would like to thank the anonymous reviewers for their detailed comments and suggestions for improvement of the original submission. The authors have incorporated most of their suggestions and of course take responsibility for any remaining flaws.

## References

- Adler, I., & Cosares, S. (1991). A strongly polynomial algorithm for a special class of linear programs.
*Operations Research*,*39*, 955–960.CrossRefGoogle Scholar - Bixby, R. E. (1994). Progress in linear programming.
*ORSA Journal on Computing*,*6*(1), 15–22.CrossRefGoogle Scholar - Borgwardt, K. H. (1987).
*The simplex method: A probabilistic analysis*. Berlin: Springer.CrossRefGoogle Scholar - Cerkinov, R. N. (1961). The solution of linear programming problems by elimination of unknowns.
*Doklady Akademii Nauk*,*139*, 1314–1317.Google Scholar - Chandru, V., & Kochar, B. S. (1985).
*A class of algorithms for linear programming*, Research Memorandum No. 85-14, Purdue University.Google Scholar - Chandru, V., & Kochar, B. S. (1986).
*Exploiting special structures using a variant of Karmarkar’s algorithm*, Research Memorandum No. 86-10, School of Industrial Engineering, Purdue University.Google Scholar - Chandru, V. (1993). Variable elimination in linear constraints.
*The Computer Journal*,*36*(5), 463–472.CrossRefGoogle Scholar - Charnes, A. (1952). Optimality and degeneracy in linear programming.
*Econometrica*,*20*, 160–170.CrossRefGoogle Scholar - Cohen, E., & Megiddo, N. (1991). Improved algorithms for linear inequalities with two variables per inequality. In
*Proceedings of the twenty third symposium on theory of computing*, New Orleans (pp. 145–155).Google Scholar - Cohen, E., & Megiddo, N. (1993). New algorithms for generalized network flows, Revised. In D. Dolev, Z. Galil, & M. Rodeh (Eds.),
*Proceedings of the 1st Israeli symposium on the theory of computing and systems*(pp. 103–114).Google Scholar - Cohen, E., & Megiddo, N. (1994). Improved algorithms for linear inequalities with two variables per inequality, Extended Abstract.
*SIAM Journal of Computing*,*23*, 1313–1347.CrossRefGoogle Scholar - Cottle, R. W., & Veinott, A. F, Jr. (1972). Polyhedral sets having a least element.
*Mathematical Programming*,*3*, 238–249.CrossRefGoogle Scholar - Dantzig, G. B. (1951). Maximization of a linear function of variables subject to linear inequalities. In C. Koopmans (Ed.),
*Activity analysis of production and allocation*(pp. 339–347). New York: Wiley.Google Scholar - Dantzig, G. B. (1963).
*Linear programming and extensions*. Princeton: Princeton University Press.CrossRefGoogle Scholar - Dantzig, G. B., & Eaves, B. C. (1973). Fourier–Motzkin elimination and its dual.
*Journal of Combinatorial Theory (A)*,*14*, 288–297.CrossRefGoogle Scholar - Dantzig, G. B., Orden, A., & Wolfe, P. (1955). The generalized simplex method for minimizing a linear form under linear inequality restraints.
*Pacific Journal of Mathematics*,*5*, 183–195.CrossRefGoogle Scholar - Fourier, L. B. J. (1826).
*Reported in: Analyse des travaux de l’Academic Royale des Sciences. pendant l’annee*(1823), Partie mathematique, Histoire de l’Academie Royale des Sciences de l’Institut de France (Vol. 6, pp. xxix–xli).Google Scholar - Fourier, L. B. J. (1827).
*Reported in: Analyse des travaux de l’Academic Royale des Sciences. pendant l’annee*(1824), Partie mathematique, Histoire de l’Academie Royale des Sciences de l’Institut de France (Vol. 7, pp. xlviii–lv) (Partial English Translation in: D.A. Kohler, Translation of a report by Fourier on his work on linear inequalities. Opsearch, Vol. 10, pp. 38–42, 1973)Google Scholar - Grötschel, M., Lovász, L., & Schrijver, A. (1982). The ellipsoid method and its consequences in combinatorial optimization.
*Combinatorica*,*1*, 169–197.CrossRefGoogle Scholar - Grötschel, M., Lovász, L., & Schrijver, A. (1988).
*Geometric algorithms and combinatorial optimization*. Berlin: Springer.CrossRefGoogle Scholar - Haĉijan, L. G. (1979). A polynomial algorithm in linear programming.
*Soviet Mathematics Doklady*,*20*, 191–194.Google Scholar - Haimovich, M. (1983).
*The simplex method is very good! On the expected number of pivot steps and related properties of random linear programs*. Unpublished Manuscript.Google Scholar - Hochbaum, D. S., & Naor, J. (1994). Simple and Fast Algorithms for Linear and Integer Programs with two variables per inequality.
*SIAM Journal on Computing*,*23*(6), 1179–1192.CrossRefGoogle Scholar - Karmarkar, N. K. (1984). A new polynomial-time algorithm for linear programming.
*Combinatorica*,*4*, 373–395.CrossRefGoogle Scholar - Karp, R. M., & Papadimitriou, C. H. (1982). On linear characterizations of combinatorial optimization problems.
*SIAM Journal on Computing*,*11*, 620–632.CrossRefGoogle Scholar - Klee, V., & Minty, G. J. (1972). How good is the simplex algorithm? In O. Shisha (Ed.),
*Inequalities III*. Cambridge: Academic Press.Google Scholar - Lassez, J.-L. (1991). From LP to LP: Programming with constraints. In
*Proceedings of theoretical aspects of computer software*, Sendai.Google Scholar - Lassez, J.-L., & Maher, M. J. (1988).
*On Fourier’s algorithm for linear arithmetic constraints*, IBM Research Report, T Watson Research Center.Google Scholar - Megiddo, N. (1983). Towards a genuinely polynomial algorithm for linear programming.
*SIAM Journal on Computing*,*12*(2), 347–353.CrossRefGoogle Scholar - Megiddo, N. (1991). On finding primal- and dual-optimal bases.
*ORSA Journal on Computing*,*3*, 63–65.CrossRefGoogle Scholar - Motzkin, T. S. (1936).
*Beitrage zur theorie der linearen Ungleichungen*. Doctoral thesis, University of Base.Google Scholar - Padberg, M. W., & Rao, M. R. (1981).
*The Russian method for linear inequalities, Part III, Bounded integer programming*. New York: New York University. (preprint).Google Scholar - Saigal, R. (1995).
*Linear programming: A modern integrated analysis*. Alphen aan den Rijn: Kluwer Press.CrossRefGoogle Scholar - Schrijver, A. (1986).
*Theory of linear and integer programming*. Hoboken: Wiley.Google Scholar - Shor, N. Z. (1970). Convergence rate of the gradient descent method with dilation of the space.
*Cybernetics*,*6*, 102–108.CrossRefGoogle Scholar - Stone, Richard E., & Tovey, Craig A. (1991). The simplex and projective scaling algorithms as iteratively reweighted least squares methods.
*SIAM Review*,*33*(2), 220–237.CrossRefGoogle Scholar - Todd, M. J. (1986).
*Exploiting special structure in Karmarkar’s algorithm for linear programming*. Technical Report 707, School of Operations Research and Industrial Engineering, Cornell University.Google Scholar - Todd, M. J. (1994). Theory and practice for interior-point methods.
*ORSA Journal on Computing*,*6*(1), 28–31.CrossRefGoogle Scholar - Vaidya, P. M. (1989). Speeding-up linear programming using fast matrix multiplication. In
*Proceedings of the 30th IEEE annual symposium on foundations of computer science*(pp. 332–337).Google Scholar - Vanderbei, R. J. (2015).
*Linear programming: Foundations and extensions*(4th ed.). Berlin: Springer.Google Scholar - Williams, H. P. (1986). Fourier’s method of linear programming and its dual.
*The America Mathematical Monthly*,*93*, 681–695.CrossRefGoogle Scholar - Wolfe, P. (1963). A technique for resolving degeneracy in linear programming.
*SIAM Journal on Applied Mathematics*,*11*(205), 211.Google Scholar - Wright, S. J. (1997).
*Primal-dual interior-point methods*. Cambridge: SIAM Press.CrossRefGoogle Scholar - Ziegler, M. (1995).
*Convex polytopes*. Berlin: Springer.Google Scholar