Skip to main content
SpringerLink
Log in

Mathematical Solution Techniques — The Nonlinear World

  • Chapter
  • First Online:
Business Optimization Using Mathematical Programming

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 307))

  • 1118 Accesses

Abstract

This chapter provides some of the mathematical and algorithmic backgrounds to solve NLP and MINLP problems to local or global optimality. Covering nonlinear, continuous, or mixed integer optimization in great depth is beyond the scope of this book. Therefore only some essential aspects and ideas are introduced and some basics are presented. Readers with further interest are referred to Gill et al. (1981), Spelluci (1993), Burer & Letchford (2012) for a survey on non-convex MINLP, Belotti et al. (2013) on MINLP, and Boukouvala et al. (2016) for advances on global optimization. Special techniques for NLP problems, often used in oil or food industry, such as recursion or sequential linear programming and distributive recursion, have already been covered in Sect. 11.2.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Named after the German mathematician Ludwig Otto Hesse (1811–1874). \( \mathcal {M}(m,n)\) denotes the set of all matrices with m rows and n columns.

  2. 2.

    Named after the German mathematician Carl Gustav Jacob Jacobi (1804–1851).

  3. 3.

    Usually, f(x k + α s k) is not exactly minimized w.r.t. α. One possible heuristic is to evaluate f for α m = 2−m for m = 0,  1,  2, …, and to stop the line search when f(x k + α m s k) ≤ f(x k).

  4. 4.

    This raises the question of how to choose the size of the increments h. As illustrated by Press et al. (1992,[447]), the optimal choice of this size depends on the curvature, i.e., the second derivative of g :  ℝ →ℝ , u → g(u). Since the second derivation is, however, unknown in most cases, we refer the reader to Press et al. (1992,[447, p.180]) and their heuristic approximation \(h\approx 2\varepsilon _{g}^{1/3}u\), where ε g is the relative precision with which g(u) is calculated. For functions that are not too complicated this corresponds approximately to machine accuracy, i.e., ε g ≈ ε m.

  5. 5.

    It was only later detected that Karush (1939,[329]) had already proven the same result in his 1939 master’s thesis at the University of Chicago. In his review article Kuhn (1976,[344]) gave a historical overview of inequality-constrained optimization.

  6. 6.

    These assumptions guarantee that both the feasible region and the objective function are convex.

  7. 7.

    No algorithm is known which can solve any \(\mathcal {N}\mathcal {P}\)-complete problem in polynomial time (the solve time is bounded by a polynomial function of the problem size). It is thought that if such an algorithm were found, it would also be able to solve other \(\mathcal {N}\mathcal {P}\)-complete problems in polynomial time, e.g., the resource constrained scheduling problem presented in Sect. 10.5. See Nemhauser & Wolsey (1988) for further material on exact definition and explanation on class \(\mathcal {N}\mathcal {P}\).

  8. 8.

    Second order convergence rate implies that we double the number of accurate digits after the decimal point in each iteration. As derivatives, especially, the Hessian, are subject to numerical errors; in practice one is usually content to prove and achieve superlinear convergence.

  9. 9.

    Equations h(x, y) = 0 are replaced by the inequality pairs − δ ≤h(x, y) ≤ δ with any given δ > 0. However, this procedure should only be used if there is no other way; it is not very efficient.

References

  1. Abadie, J.: The GRG method for nonlinear programming. In: Greenberg, H.J. (ed.) Design and Implementation of Optimization Software, pp. 335–363. Sijthoff and Noordhoff, Holland (1978)

    Chapter  Google Scholar 

  2. Abadie, J., Carpenter, J.: Generalization of the Wolfe reduced gradient method to the case of nonlinear constraints. In: Fletcher, R. (ed.) Optimization, pp. 37–47. Academic Press, New York (1969)

    Google Scholar 

  3. Adjiman, C.S.J.: Global Optimization Techniques for Process Systems Engineering. PhD Dissertation, Dept. of Chemical Engineering, Princeton University, Princeton, NJ (1999)

    Google Scholar 

  4. Adjiman, C., Dallwig, S., Floudas, C., Neumaier, A.: A global optimization method, αBB, for general twice-differentiable constrained NLPs - I. Theoretical advances. Comput. Chem. Eng. 22, 1137–1158 (1998)

    Article  Google Scholar 

  5. Adjiman, C., Dallwig, S., Floudas, C., Neumaier, A.: A global optimization method, αBB, for general twice-differentiable constrained NLPs - II. Implementation and computational results. Comput. Chem. Eng. 22, 1159–1179 (1998)

    Article  Google Scholar 

  6. Al-Khayyal, F.A.: Jointly constrained bilinear programs and related problems: an overview. Comput. Math. Appl. 19, 53–62 (1990)

    Article  Google Scholar 

  7. Al-Khayyal, F.A., Falk, J.E.: Jointly constrained biconvex programming. Math. Ops. Res. 8, 273–286 (1983)

    Article  Google Scholar 

  8. Andrei, N.: Nonlinear Optimization Applications Using the GAMS Technology. Springer, New York (2013)

    Book  Google Scholar 

  9. Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming Theory and Algorithms, 2nd edn. Wily-Interscience Series in Discrete Mathematics and Optimization. Wiley, New York (1993)

    Google Scholar 

  10. Belotti, P., Kirches, C., Leyffer, S., Linderoth, J., Luedtke, J., Mahajan, A.: Mixed-integer nonlinear optimization. Acta Numer. 22, 1–131 (2013)

    Article  Google Scholar 

  11. Biegler, L.T., Grossmann, I.: Retrospective on optimization. Comput. Chem. Eng. 28, 1169–1192 (2004)

    Article  Google Scholar 

  12. Bock, H.G.: Randwertproblemmethoden zur Parameteridentifizierung in Systemen nichtlinearer Differentialgleichungen. Preprint 142, Universität Heidelberg, SFB 123. Institut für Angewandte Mathematik, Heidelberg (1987)

    Google Scholar 

  13. Bomze, I.M., Grossmann, W.: Optimierung – Theorie und Algorithmen. Wissenschaftsverlag, Mannheim (1993)

    Google Scholar 

  14. Bonami, P., Kilinç, M., Linderoth, J.: Algorithms and software for convex mixed integer nonlinear programs. In: Lee, J., Leyffer, S. (eds.) Mixed Integer Nonlinear Programming. The IMA Volumes in Mathematics and its Applications, vol. 154, pp. 1–39. Springer, New York (2012)

    Google Scholar 

  15. Boukouvala, F., Misener, R., Floudas, C.A.: Global optimization advances in mixed-integer nonlinear programming, MINLP, and constrained derivative-free optimization, CDFO. Eur. J. Oper. Res. 252(3), 701–727 (2016)

    Article  Google Scholar 

  16. Brooke, A., Kendrick, D., Meeraus, A.: GAMS - A User’s Guide (Release 2.25). Boyd & Fraser Publishing Company, Danvers, MA (1992)

    Google Scholar 

  17. Burer, S., Letchford, A.N.: Non-convex mixed-integer nonlinear programming: a survey. Surv. Oper. Res. Manage. Sci. 17(2), 97–106 (2012)

    Google Scholar 

  18. Collatz, L., Wetterling, W.: Optimierungsaufgaben, 2nd edn. Springer, Berlin (1971)

    Book  Google Scholar 

  19. Drud, A.S.: CONOPT - a large-scale GRG code. ORSA J. Comput. 6(2), 207–218 (1994)

    Article  Google Scholar 

  20. Duran, M.A., Grossmann, I.E.: An outer-approximation algorithm for a class of mixed-integer nonlinear programmes. Math. Program. 36, 307–339 (1986)

    Article  Google Scholar 

  21. Emet, S., Westerlund, T.: Solving a dynamic separation problem using MINLP techniques. Appl. Numer. Math. 58(12), 395–406 (2008)

    Article  Google Scholar 

  22. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, Chichester (1987)

    Google Scholar 

  23. Floudas, C.A.: Nonlinear and Mixed-Integer Optimization : Fundamentals and Applications. Oxford University Press, Oxford (1995)

    Book  Google Scholar 

  24. Floudas, C.A.: Deterministic Global Optimization: Theory, Methods and Applications. Kluwer Academic Publishers, Dordrecht (2000)

    Book  Google Scholar 

  25. Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Glob. Optim. 45, 3–38 (2009)

    Article  Google Scholar 

  26. Floudas, C.A., Pardalos, P.M. (eds.): Frontiers in Global Optimization. Kluwer Academic Publishers, Dordrecht (2004)

    Google Scholar 

  27. Floudas, C.A., Akrotirianakis, I.G., Caratzoulas, S., Meyer, C.A., Kallrath, J.: Global optimization in the 21st century: advances and challenges for problems with nonlinear dynamics. Comput. Chem. Eng. 29, 1185–1202 (2005)

    Article  Google Scholar 

  28. Geoffrion, A.M.: Generalized benders decomposition. J. Optim. Theor. Appl. 10, 237–260 (1972)

    Article  Google Scholar 

  29. Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic Press, London (1981)

    Google Scholar 

  30. Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP Algorithm for Large-scale Constrained Optimization. Numerical analysis report 97-2, Department of Mathematics, University of California, San Diego, San Diego, La Jolla, CA (1997)

    Google Scholar 

  31. Grossmann, I.E.: Review of nonlinar mixed-integer and disjunctive programming techniques. Optim. Eng. 3, 227–252 (2002)

    Article  Google Scholar 

  32. Gupta, O.K., Ravindran, V.: Branch and bound experiments in convex nonlinear integer programming. Manage. Sci. 31, 1533–1546 (1985)

    Article  Google Scholar 

  33. Hendrix, E.M.T., G.-Tóth, B.: Introduction to Nonlinear and Global Optimization. Springer Optimization and Its Applications, vol. 37. Springer, New York, NY (2010)

    Google Scholar 

  34. Hertz, D.: The extreme eigenvalues and stability of real symmetric interval matrices. IEEE Trans. Autom. Control 37, 532–535 (1992)

    Article  Google Scholar 

  35. Horst, R., Pardalos, P.M. (eds.): Handbook of Global Optimization. Kluwer Academic Publishers, Dordrecht (1995)

    Google Scholar 

  36. Horst, R., Pardalos, P.M., Thoai, N.V.: Introduction to Global Optimization, 2nd edn. Kluwer Academic Publishers, Dordrecht (2000)

    Book  Google Scholar 

  37. Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches, 3rd edn. Springer, New York (1996)

    Book  Google Scholar 

  38. Karush, W.: Minima of Functions of Several Variables with Inequalities as Side Constraints. Master thesis, Department of Mathematics, University of Chicago, Chicago (1939)

    Google Scholar 

  39. Kearfott, R.B.: Rigorous Global Search: Continuous Problems. Kluwer Academic Publishers, Dordrecht (1996)

    Book  Google Scholar 

  40. Kelley, J.E.: The cutting plane method for solving convex programs. J. SIAM 8(4), 703–712 (1960)

    Google Scholar 

  41. Kilinç, M.R., Sahinidis, N.V.: State of the Art in mixed-integer nonlinear optimization, chap. 21, pp. 273–292. SIAM, Philadelphia (2017)

    Google Scholar 

  42. Kronqvist, J., Bernal, D.E., Lundell, A., Grossmann, I.E.: A review and comparison of solvers for convex MINLP. Optim. Eng. 20(2), 397–455 (2019)

    Article  Google Scholar 

  43. Kuhn, H.: Nonlinear programming: a historical view. In: Cottle, R., Lemke, C. (eds.) Nonlinear Programming. SIAM-AMS Proceedings, vol. 9, pp. 1–26. American Mathematical Society, Providence, RI (1976)

    Google Scholar 

  44. Kuhn, H.W., Tucker, A.W.: Nonlinear programming. In: Neumann, J. (ed.) Proceedings Second Berkeley Symposium on Mathematical Statistics and Probability, pp. 481–492. University of California, Berkeley, CA (1951)

    Google Scholar 

  45. Lasdon, L.S., Waren, A.D.: Generalized reduced gradient method for linearly and nonlinearly constrained programming. In: Greenberg, H.J. (ed.) Design and Implementation of Optimization Software, pp. 363–397. Sijthoff and Noordhoff, Alphen aan den Rijn (1978)

    Google Scholar 

  46. Lasdon, L.S., Waren, A.D., Jain, A., Ratner, M.: Design and testing of a generalized reduced gradient code for nonlinear programming. ACM Trans. Math. Softw. 4, 34–50 (1978)

    Article  Google Scholar 

  47. Leyffer, S.: Deterministic Methods for Mixed Integer Nonlinear Programming. PhD Thesis, Department of Mathematics and Computer Science, University of Dundee, Dundee (1993)

    Google Scholar 

  48. Liberti, L., Maculan, N. (eds.): Global Optimization: From Theory to Implementation. Nonconvex Optimization and Its Applications, vol. 84, pp. 223–232. Springer, New York (2006)

    Google Scholar 

  49. Maranas, C.D., Floudas, C.A.: Global minimum potential energy confirmations of small molecules. J. Global Optim. 4, 135–170 (1994)

    Article  Google Scholar 

  50. McCormick, G.P.: Computation of global solutions to factorable nonconvex programs: part I - convex underestimations problems. Math. Program. 10, 147–175 (1976)

    Article  Google Scholar 

  51. Misener, R., Floudas, C.: GloMIQO: global mixed-integer quadratic optimizer. J. Glob. Optim. 1–48 (2012). http://dx.doi.org/10.1007/s10898-012-9874-7

  52. Murtagh, B.A., Saunders, M.A.: Large-scale linearly constrained optimization. Math. Program. 14, 41–72 (1978)

    Article  Google Scholar 

  53. Murtagh, B.A., Saunders, M.A.: A projected Lagrangian algorithm and its implementation for sparse nonlinear constraints. Math. Program. Study (Algorithm for Constrained Minimization of Smooth Nonlinear Function) 16, 84–117 (1982)

    Google Scholar 

  54. Muts, P., Nowak, I., Hendrix, E.M.T.: The decomposition-based outer approximation algorithm for convex mixed-integer nonlinear programming. J. Glob. Optim. 77(1), 75–96 (2020)

    Article  Google Scholar 

  55. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comp. J. 7, 308–313 (1965)

    Article  Google Scholar 

  56. Nowak, I.: Relaxation and Decomposition Methods for Mixed Integer Nonlinear Programming. Birkhäuser, Basel (2005)

    Google Scholar 

  57. Nowak, I., Muts, P., Hendrix, E.M.T.: Multi-tree decomposition methods for large-scale mixed integer nonlinear optimization. In: Velásquez-Bermúdez, J.M., Khakifirooz, M., Fathi, M. (eds.) Large Scale Optimization in Supply Chains and Smart Manufacturing: Theory and Applications, pp. 27–58. Springer International Publishing, Cham (2019)

    Chapter  Google Scholar 

  58. Papadimitriou, C.H., Steiglitz, K.: Combinatorial Optimization: Algorithms and Complexity. Prentice Hall, Englewood Cliffs, NJ (1982)

    Google Scholar 

  59. Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Numerical Recipes - The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge (1992)

    Google Scholar 

  60. Ratschek, H., Rokne, J.: Interval methods. In: Horst, R., Pardalos, P.M. (eds.) Handbook of Global Optimization, pp. 751–828. Kluwer Academic Publishers, Dordrecht (1995)

    Chapter  Google Scholar 

  61. Ravindran, A., Phillips, D.T., Solberg, J.J.: Operations Research. Principles and Practice. Wiley, New York (1987)

    Google Scholar 

  62. Robinson, S.M.: A quadratically convergent algorithm for general nonlinear for programming problems. Math. Program. 3, 145–156 (1972)

    Article  Google Scholar 

  63. Sahinidis, N.V.: Mixed-integer nonlinear programming 2018. Optim. Eng. 20, 301–306 (2018)

    Article  Google Scholar 

  64. Schweiger, C.A., Rojnuckarin, A., Floudas, C.A.: MINOPT: a software package for mixed-integer nonlinear optimization. Dept. of Chemical Engineering, Princeton University, Princeton, NJ (1996)

    Google Scholar 

  65. Spelluci, P.: Numerische Verfahren der nichtlinearen Optimierung. Birkhäuser, Basel (1993)

    Google Scholar 

  66. Spendley, W., Hext, G.R., Himsworth, F.R.: Sequential application of simplex designs in optimisation and evolutionary operation. Technometrics 4, 441–461 (1962)

    Article  Google Scholar 

  67. Stoer, J.: Foundations of recursive quadratic programming methods for solving nonlinear programs. In: Schittkowski, K. (ed.) Computational Mathematical Programming. NATO ASI Series, vol. 15. Springer, Heidelberg (1985)

    Google Scholar 

  68. Tawarmalani, M., Sahinidis, N.V.: Convexification and global optimization in continuous and mixed-integer nonlinear programming: theory, algorithms, software, and applications. In: Nonconvex Optimization And Its Applications, vol. 65. Kluwer Academic Publishers, Dordrecht (2002)

    Google Scholar 

  69. Tawarmalani, M., Sahinidis, N.V.: Global optimization of mixed integer nonlinear programs: a theoretical and computational study improve MIP solutions. Math. Program. 99, 563–591 (2004)

    Article  Google Scholar 

  70. Trespalacios, F., Grossmann, I.E.: Review of mixed-integer nonlinear and generalized disjunctive programming methods. Chem. Ing. Tech. 86, 991–1012 (2014)

    Article  Google Scholar 

  71. Viswanathan, J., Grossmann, I.E.: A combined penalty function and outer-approximation method for MINLP optimization. Comp. Chem. Eng. 14(7), 769–782 (1990)

    Article  Google Scholar 

  72. Wächter, A., Biegler, L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106, 25–57 (2006)

    Article  Google Scholar 

  73. Werner, J.: Numerische Mathematik. Vieweg, Wiesbaden, Deutschland (1992)

    Book  Google Scholar 

  74. Westerlund, T., Pörn, R.: Solving pseudo-convex mixed integer problems by cutting plane techniques. Optim. Eng. 3, 253–280 (2002)

    Article  Google Scholar 

  75. Westerlund, T., Petterson, F.: An extended cutting plane method for solving convex MINLP problems. Comput. chem. Eng. Sup. 19, S131–136 (1995)

    Article  Google Scholar 

  76. Westerlund, T., Skrifvars, H., Harjunkoski, I., Pörn, R.: An extended cutting plane method for solving a class of non-convex MINLP problems. Comput. Chem. Eng. 22, 357–365 (1998)

    Article  Google Scholar 

  77. Westerlund, T., Eronen, V., Mäkelä, M.M.: On solving generalized convex MINLP problems using supporting hyperplane techniques. J. Glob. Optim. 71(4), 987–1011 (2018)

    Article  Google Scholar 

  78. Wright, S.: Primal-Dual Interior-Point Methods. Society for Industrial and Applied Mathematics, Philadelphia, PA (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Astronomy, University of Florida, Gainesville, FL, USA

    Josef Kallrath

Authors
  1. Josef Kallrath
    View author publications

    You can also search for this author in PubMed Google Scholar

12.1 Electronic Supplementary Material

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kallrath, J. (2021). Mathematical Solution Techniques — The Nonlinear World. In: Business Optimization Using Mathematical Programming. International Series in Operations Research & Management Science, vol 307. Springer, Cham. https://doi.org/10.1007/978-3-030-73237-0_12

Download citation

Publish with us

Policies and ethics

Search

Navigation