Annals of Operations Research

, Volume 153, Issue 1, pp 257–296 | Cite as

Complexity and algorithms for nonlinear optimization problems

  • Dorit S. Hochbaum


Nonlinear optimization algorithms are rarely discussed from a complexity point of view. Even the concept of solving nonlinear problems on digital computers is not well defined. The focus here is on a complexity approach for designing and analyzing algorithms for nonlinear optimization problems providing optimal solutions with prespecified accuracy in the solution space. We delineate the complexity status of convex problems over network constraints, dual of flow constraints, dual of multi-commodity, constraints defined by a submodular rank function (a generalized allocation problem), tree networks, diagonal dominant matrices, and nonlinear knapsack problem’s constraint. All these problems, except for the latter in integers, have polynomial time algorithms which may be viewed within a unifying framework of a proximity-scaling technique or a threshold technique. The complexity of many of these algorithms is furthermore best possible in that it matches lower bounds on the complexity of the respective problems.

In general nonseparable optimization problems are shown to be considerably more difficult than separable problems. We compare the complexity of continuous versus discrete nonlinear problems and list some major open problems in the area of nonlinear optimization.


Nonlinear optimization Convex network flow Strongly polynomial algorithms Lower bounds on complexity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ahuja, R. K., & Orlin, J. B. (2001b). Inverse optimization. Operations Research, 49, 771–783. CrossRefGoogle Scholar
  2. Ahuja, R. K., Batra, J. L., & Gupta, S. K. (1984). A parametric algorithm for the convex cost network flow and related problems. European Journal of Operational Research, 16, 222–235. CrossRefGoogle Scholar
  3. Ahuja, R. K., Hochbaum, D. S., & Orlin, J. B. (2003). Solving the convex cost integer dual network flow problem. Management Science, 49, 950–964. CrossRefGoogle Scholar
  4. Ahuja, R. K., Hochbaum, D. S., & Orlin, J. B. (2004). A cut based algorithm for the nonlinear dual of the minimum cost network flow problem. Algorithmica, 39, 189–208. CrossRefGoogle Scholar
  5. Ahuja, R. K., Magnanti, T. L., & Orlin, J. B. (1993). Network flows: Theory, algorithms and applications. New Jersey: Prentice Hall. Google Scholar
  6. Baldick, R. (1991). A unification of polynomially solvable cases of integer ‘non-separable’ quadratic optimization. Lawrence Berkeley Laboratory manuscript. Google Scholar
  7. Baldick, R., & Wu, F. F. (1990). Efficient integer optimization algorithms for optimal coordination of capacitors and regulators. IEEE Transactions on Power Systems, 5, 805–812. CrossRefGoogle Scholar
  8. Barahona, F. (1986). A solvable case of quadratic 0-1 programming. Discrete Applied Mathematics, 13, 23–28. CrossRefGoogle Scholar
  9. Barlow, R. E., Bartholomew, D. J., Bremer, J. M., & Brunk, H. D. (1972). Statistical inference under order restrictions. New York: Wiley. Google Scholar
  10. Blum, M., Floyd, R. W., Pratt, V. R., Rivest, R. L., & Tarjan, R. E. (1972). Time bounds for selection. Journal of Computer Systems Science, 7, 448–461. CrossRefGoogle Scholar
  11. Brucker, P. (1984). An O(n) algorithm for quadratic knapsack problems. Operations Research Letters, 3, 163–166. CrossRefGoogle Scholar
  12. Burton, D., & Toint, Ph. L. (1992). On an instance of the inverse shortest paths problem. Mathematical Programming, 53, 45–61. CrossRefGoogle Scholar
  13. Burton, D., & Toint, Ph. L. (1994). On the use of an inverse shortest paths algorithm for recovering linearly correlated costs. Mathematical Programming, 63, 1–22. CrossRefGoogle Scholar
  14. Busacker, R. G., & Gowen, P. J. (1961) A procedure for determining minimal-cost network flow patterns. Operational Research Office, John Hopkins University, Baltimore, MD. Google Scholar
  15. Cosares, S., & Hochbaum, D. S. (1994). A strongly polynomial algorithm for the quadratic transportation problem with fixed number of suppliers. Mathematics of Operations Research, 19, 94–111. Google Scholar
  16. Dantzig, G. B. (1963). Linear programming and extensions. New Jersey: Princeton University Press. Google Scholar
  17. Dennis, J. B. (1959). Mathematical programming and electrical networks. In Technology press research monographs (pp. 74–75). New York: Technology Press and Wiley. Google Scholar
  18. Edmonds, J., & Karp, R. M. (1972). Theoretical improvements in algorithmic efficiency for network flow problems. Journal of ACM, 19, 248–264. CrossRefGoogle Scholar
  19. Erickson, R. E., Monma, C. L., & Veinott, A. F. (1987). Send-and-split method for minimum-concave-cost network flows. Mathematics of Operations Research, 12, 634–664. Google Scholar
  20. Federgruen, A., & Groenevelt, H. (1986a). The greedy procedure for resource allocation problems: Necessary and sufficient conditions for optimality. Operations Research, 34, 909–918. Google Scholar
  21. Federgruen, A., & Groenevelt, H. (1986b). Optimal flows in networks with multiple sources and sinks, with applications to oil and gas lease investment programs. Operations Research, 34, 218–225. Google Scholar
  22. Frederickson, G. N., & Johnson, D. B. (1982). The complexity of selection and rankings in X+Y and matrices with sorted columns. Journal of Computing System Science, 24, 197–208. CrossRefGoogle Scholar
  23. Fourer, R. (1988). A simplex algorithm for piecewise-linear programming: Finiteness, feasibility and degeneracy. Mathematical Programming, 41, 281–316. CrossRefGoogle Scholar
  24. Gale, D. (1957). A theorem of flows in networks. Pacific Journal of Mathematics, 7, 1073–1082. Google Scholar
  25. Gallo, G., Grigoriadis, M. D., & Tarjan, R. E. (1989). A fast parametric maximum flow algorithm and applications. SIAM Journal of Computing, 18, 30–55. CrossRefGoogle Scholar
  26. Garey, M., & Johnson, D. (1979). Computers and intractability, a guide to the theory of NP-completeness. New York: Freeman. Google Scholar
  27. Goldberg, A. V., & Tarjan, R. E. (1988). A new approach to the maximum flow problem. Journal of the ACM, 35, 921–940. CrossRefGoogle Scholar
  28. Granot, F., & Skorin-Kapov, J. (1990). Some proximity and sensitivity results in quadratic integer programming. Mathematical Programming, 47, 259–268. CrossRefGoogle Scholar
  29. Guisewite, G., & Pardalos, P. M. (1990). Minimum concave cost network flow problems: Applications, complexity, and algorithms. Annals of Operations Research, 25, 75–100. CrossRefGoogle Scholar
  30. Hansen, P., & Simeone, B. (1986). Unimodular functions. Discrete Applied Mathematics, 14, 269–281. CrossRefGoogle Scholar
  31. Hochbaum, D. S. (1989). On a polynomial class of nonlinear optimization problems. Manuscript, U.C. Berkeley. Google Scholar
  32. Hochbaum, D. S. (1993). Polynomial algorithms for convex network optimization. In D. Du, M. Pardalos (Eds.), Network optimization problems: algorithms, complexity and applications (pp. 63–92). Singapore: World Scientific. Google Scholar
  33. Hochbaum, D. S. (1994). Lower and upper bounds for allocation problems. Mathematics of Operations Research, 19, 390–409. Google Scholar
  34. Hochbaum, D. S. (1995). A nonlinear knapsack problem. Operations Research Letters, 17, 103–110. CrossRefGoogle Scholar
  35. Hochbaum, D. S. (1998). The pseudoflow algorithm for the maximum flow problem. Manuscript, UC Berkeley, revised 2003. Extended abstract in: Boyd & Rios-Mercado (Eds.), Lecture notes in computer science: Vol. 1412. The pseudoflow algorithm and the pseudoflow-based simplex for the maximum flow problem. Proceedings of IPCO98 (pp. 325–337), Bixby, June 1998. New York: Springer. Google Scholar
  36. Hochbaum, D. S. (2001). An efficient algorithm for image segmentation, Markov random fields and related problems. Journal of the ACM, 48, 686–701. CrossRefGoogle Scholar
  37. Hochbaum, D. S. (2002). The inverse shortest paths problem. Manuscript, UC Berkeley. Google Scholar
  38. Hochbaum, D. S. (2003). Efficient algorithms for the inverse spanning tree problem. Operations Research, 51, 785–797. CrossRefGoogle Scholar
  39. Hochbaum, D. S. (2005). Complexity and algorithms for convex network optimization and other nonlinear problems. 4OR, 3, 171–216. CrossRefGoogle Scholar
  40. Hochbaum, D. S., & Hong, S. P. (1995). About strongly polynomial time algorithms for quadratic optimization over submodular constraints. Mathematical Programming, 69, 269–309. Google Scholar
  41. Hochbaum, D. S., & Hong, S. P. (1996). On the complexity of the production-transportation problem. SIAM Journal on Optimization, 6, 250–264. CrossRefGoogle Scholar
  42. Hochbaum, D. S., & Queyranne, M. (2003). The convex cost closure problem. SIAM Journal on Discrete Mathematics, 16, 192–207. CrossRefGoogle Scholar
  43. Hochbaum, D. S., & Seshadri, S. (1993). The empirical performance of a polynomial algorithm for constrained nonlinear optimization. Annals of Operations Research, 43, 229–248. CrossRefGoogle Scholar
  44. Hochbaum, D. S., & Shanthikumar, J. G. (1990). Convex separable optimization is not much harder than linear optimization. Journal of the ACM, 37, 843–862. CrossRefGoogle Scholar
  45. Hochbaum, D. S., Shamir, R., & Shanthikumar, J. G. (1992). A polynomial algorithm for an integer quadratic nonseparable transportation problem. Mathematical Programming, 55, 359–372. CrossRefGoogle Scholar
  46. Hoffman, A. J. (1960). Some recent applications of the theory of linear inequalities to extremal combinatorial analysis. In R. Bellman, M. Hall Jr. (Eds.), Proceedings of Symposia in Applied Mathematics: Vol. X. Combinatorial analysis (pp. 113–127). Providence: American mathematical Society. Google Scholar
  47. Ibaraki, T., & Katoh, N. (1988). Resource allocation problems: Algorithmic approaches. Boston: MIT. Google Scholar
  48. Iri, M. (1960). A new method of solving transportation network problems. Journal of the Operations Research Society of Japan, 3, 27–87. Google Scholar
  49. Jewell, W. S. (1958). Optimal flow through networks. Technical report No. 8, Operations research Center, MIT, Cambridge. Google Scholar
  50. Kapoor, S., & Vaidya, P. M. (1986). Fast algorithms for convex quadratic programming and multicommodity flows. In Proceedings of the 18th symposium on theory of computing (pp. 147–159). Google Scholar
  51. Karzanov, A. V., & McCormick, S. T. (1997). Polynomial methods for separable convex optimization in unimodular linear spaces with applications. SIAM Journal on Computing, 26, 1245–1275. CrossRefGoogle Scholar
  52. Knuth, D. (1973). The art of computer programming: Vol. 3. Sorting and searching. Reading: Addison Wesley. Google Scholar
  53. Kozlov, M. K., Tarasov, S. P., & Khachian, L. G. (1979). Polynomial solvability of convex quadratic programming. Doklady Akad. Nauk SSSR, 5, 1051–1053 (Translated in Soviet Mathematics Doklady 20 (1979), 1108–1111). Google Scholar
  54. Lawler, E. (1979). Fast approximation algorithms for knapsack problems. Mathematics of Operations Research, 4, 339–356. Google Scholar
  55. Mansour, Y., Schieber, B., & Tiwari, P. (1991). Lower bounds for computations with the floor operation. SIAM Journal on Computing, 20, 315–327. CrossRefGoogle Scholar
  56. Megiddo, N., & Tamir, A. (1993). Linear time algorithms for some separable quadratic programming problems. Operations Research Letters, 13, 203–211. CrossRefGoogle Scholar
  57. Minoux, M. (1984). A polynomial algorithm for minimum quadratic cost flow problems. European Journal of Operational Research, 18, 377–387. CrossRefGoogle Scholar
  58. Minoux, M. (1986). Solving integer minimum cost flows with separable convex cost objective polynomially. Mathematical Programming Study, 26, 237–239. Google Scholar
  59. Minoux, M. (1986). Mathematical programming, theory and algorithms. Wiley: New York, Chaps. 5, 6. Google Scholar
  60. Monteiro, R. D. C., & Adler (1989). Interior path following primal-dual algorithms. Part II: Convex quadratic programming. Mathematical Programming, 44, 43–66. CrossRefGoogle Scholar
  61. Moriguchi, S., & Shioura, A. (2004). On Hochbaum’s proximity-scaling algorithm for the general resource allocation problem. Mathematics of Operations Research, 29, 394–397. CrossRefGoogle Scholar
  62. Nemirovsky, A. S., & Yudin, D. B. (1983). Problem complexity and method efficiency in optimization. New York: Wiley. Google Scholar
  63. Papadimitiou, C. H., & Steiglitz, K. (1982). Combinatorial optimization: algorithms and complexity. New Jersey: Prentice Hall. Google Scholar
  64. Picard, J. C. (1976). Maximal closure of a graph and applications to combinatorial problems. Management Science, 22, 1268–1272. CrossRefGoogle Scholar
  65. Pinto, Y., & Shamir, R. (1994). Efficient algorithms for minimum-cost flow problems with piecewise-linear convex costs. Algorithmica, 11(3), 256–276. CrossRefGoogle Scholar
  66. Radzik, T. (1993). Parametric flows, Weighted means of cuts, and fractional combinatorial optimization. In P.M. Pardalos (Ed.), Complexity in numerical optimization (pp. 351–386). Singapore: World Scientific. Google Scholar
  67. Renegar, J. (1987). On the worst case arithmetic complexity of approximation zeroes of polynomials. Journal of Complexity, 3, 90–113. CrossRefGoogle Scholar
  68. Rote, G., & Zachariasen, M. (2007, to appear). Matrix scaling by network flow. In Proceedings of SODA07. Google Scholar
  69. Sahni, S. (1974). Computationally related Problems. SIAM Journal on Computing, 3, 262–279. CrossRefGoogle Scholar
  70. Shub, M., & Smale, S. (1996). Computational complexity: On the geometry of polynomials and a theory of cost, II. SIAM Journal on Computing, 15, 145–161. CrossRefGoogle Scholar
  71. Sun, J., Tsai, K. -H., & Qi, L. (1993). A simplex method for network programs with convex separable piecewise linear costs and its application to stochastic transshipment problems. In D. Du & P. M. Pardalos (Eds.), Network optimization problems: Algorithms, complexity and applications (pp. 283–300). Singapore: World Scientific. Google Scholar
  72. Tamir, A. (1993). A strongly polynomial algorithm for minimum convex separable quadratic cost flow problems on series-parallel networks. Mathematical Programming, 59, 117–132. CrossRefGoogle Scholar
  73. Tardos, E. (1985). A strongly polynomial minimum cost circulation algorithm. Combinatorica, 5, 247–255. CrossRefGoogle Scholar
  74. Tardos, E. (1986). A strongly polynomial algorithm to solve combinatorial linear programs. Operations Research, 34, 250–256. Google Scholar
  75. Värbrand, P., Tuy, H., Ghannadan, S., & Migdalas, A. (1995). The minimum concave cost network flow problems with fixed number of sources and non-linear arc costs. Journal of Global Optimisation, 6, 135–151. CrossRefGoogle Scholar
  76. Värbrand, P., Tuy, H., Ghannadan, S., & Migdalas, A. (1996). A strongly polynomial algorithm for a concave production-transportation problem with a fixed number of non-linear variables. Mathematical Programming, 72, 229–258. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Industrial Engineering and Operations Research and Walter A. Haas School of BusinessUniversity of CaliforniaBerkeleyUSA

Personalised recommendations