Journal of Global Optimization

, Volume 67, Issue 4, pp 687–729 | Cite as

Differentiable McCormick relaxations

  • Kamil A. Khan
  • Harry A. J. Watson
  • Paul I. Barton
Article

Abstract

McCormick’s classical relaxation technique constructs closed-form convex and concave relaxations of compositions of simple intrinsic functions. These relaxations have several properties which make them useful for lower bounding problems in global optimization: they can be evaluated automatically, accurately, and computationally inexpensively, and they converge rapidly to the relaxed function as the underlying domain is reduced in size. They may also be adapted to yield relaxations of certain implicit functions and differential equation solutions. However, McCormick’s relaxations may be nonsmooth, and this nonsmoothness can create theoretical and computational obstacles when relaxations are to be deployed. This article presents a continuously differentiable variant of McCormick’s original relaxations in the multivariate McCormick framework of Tsoukalas and Mitsos. Gradients of the new differentiable relaxations may be computed efficiently using the standard forward or reverse modes of automatic differentiation. Extensions to differentiable relaxations of implicit functions and solutions of parametric ordinary differential equations are discussed. A C++ implementation based on the library MC++ is described and applied to a case study in nonsmooth nonconvex optimization.

Keywords

Nonconvex optimization Convex underestimators McCormick relaxations Implicit functions 

Mathematics Subject Classification

49M20 90C26 65G40 26B25 

References

  1. 1.
    Achterberg, T.: SCIP: solving constraint integer programs. Math. Program. Comput. 1, 1–41 (2009)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Achterberg, T., Berthold, T., Koch, T., Wolter, K.: Constraint integer programming: a new approach to integrate CP and MIP. In: Proceedings of the Fifth International Conference on Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, pp. 6–20. Paris (2008)Google Scholar
  3. 3.
    Adjiman, C.S., Dallwig, S., Floudas, C.A., Neumaier, A.: A global optimization method, \(\alpha \)BB, for general twice-differentiable constrained NLPs—I. Theoretical advances. Comput. Chem. Eng. 22, 1137–1158 (1998)CrossRefGoogle Scholar
  4. 4.
    Alefeld, G., Mayer, G.: Interval analysis: theory and applications. J. Comput. Appl. Math. 121, 421–464 (2000)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms, 3rd edn. Wiley, Hoboken (2006)CrossRefMATHGoogle Scholar
  6. 6.
    Beckers, M., Mosenkis, V., Naumann, U.: Adjoint mode computation of subgradients for McCormick relaxations. In: Forth, S., Hovland, P., Phipps, E., Utke, J., Walther, A. (eds.) Recent Advances in Algorithmic Differentiation, pp. 103–113. Springer, Berlin (2012)CrossRefGoogle Scholar
  7. 7.
    Belotti, P.: COUENNE: A user’s manual. https://projects.coin-or.org/Couenne (2006)
  8. 8.
    Bertsekas, D.P.: Nondifferentiable optimization via approximation. In: Balinski, M., Wolfe, P. (eds.) Mathematical Programming Study 3, pp. 1–25. North-Holland Publishing Company, Amsterdam (1975)Google Scholar
  9. 9.
    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)MATHGoogle Scholar
  10. 10.
    Bompadre, A., Mitsos, A.: Convergence rate of McCormick relaxations. J. Glob. Optim. 52, 1–28 (2012)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)CrossRefMATHGoogle Scholar
  12. 12.
    Broyden, C.G., Dennis Jr., J.E., Moré, J.J.: On the local and superlinear convergence of quasi-Newton methods. J. Inst. Math. Appl. 12, 223–245 (1973)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Chachuat, B.: MC++: a toolkit for bounding factorable functions, v1.0. Retrieved 2 July 2014 https://projects.coin-or.org/MCpp (2014)
  14. 14.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1990)CrossRefMATHGoogle Scholar
  15. 15.
    Coddington, E.A., Levinson, N.: Theory of Ordinary Differential Equations. McGraw Hill Co., Inc., New York (1955)MATHGoogle Scholar
  16. 16.
    Du, K., Kearfott, R.B.: The cluster problem in multivariate global optimization. J. Glob. Optim. 5, 253–265 (1994)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. 2. Springer, New York (2003)MATHGoogle Scholar
  18. 18.
    Feehery, W.F., Tolsma, J.E., Barton, P.I.: Efficient sensitivity analysis of large-scale differential-algebraic systems. Appl. Numer. Math. 25, 41–54 (1997)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Gabriel, S.A., Moré, J.J.: Smoothing of Mixed Complementarity Problems. Preprint MCS-P541-0995, Argonne National Laboratory (1995)Google Scholar
  20. 20.
    Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev. 47(1), 99–131 (2002)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Griewank, A., Rabier, P.J.: On the smoothness of convex envelopes. Trans. Am. Math. Soc. 322, 691–709 (1990)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Other Titles in Applied Mathematics, 2nd edn. SIAM, Philadelphia (2008)CrossRefMATHGoogle Scholar
  23. 23.
    Grossmann, I.E., Yeomans, H., Kravanja, Z.: A rigorous disjunctive optimization model for simultaneous flowsheet optimization and heat integration. Comput. Chem. Eng. 22(98), 157–164 (1998)CrossRefGoogle Scholar
  24. 24.
    Hartman, P.: Ordinary Differential Equations, 2nd edn. SIAM, Philadelphia (2002)CrossRefMATHGoogle Scholar
  25. 25.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I: Fundamentals. A Series of Comprehensive Studies in Mathematics. Springer, Berlin (1993)MATHGoogle Scholar
  26. 26.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II: Advanced Theory and Bundle Methods. A Series of Comprehensive Studies in Mathematics. Springer, Berlin (1993)MATHGoogle Scholar
  27. 27.
    Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches, 2nd edn. Springer, Berlin (1993)CrossRefMATHGoogle Scholar
  28. 28.
    Kesavan, P., Allgor, R.J., Gatzke, E.P., Barton, P.I.: Outer approximation algorithms for separable nonconvex mixed-integer nonlinear programs. Math. Program. Ser. A 100, 517–535 (2004)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Khan, K.A.: Sensitivity analysis for nonsmooth dynamic systems. Ph.D. thesis, Massachusetts Institute of Technology (2015)Google Scholar
  30. 30.
    Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics. Springer, Berlin (1985)CrossRefMATHGoogle Scholar
  31. 31.
    Lemaréchal, C., Strodiot, J.J., Bihain, A.: On a bundle algorithm for nonsmooth optimization. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming 4. Academic Press, New York (1981)Google Scholar
  32. 32.
    Li, X., Tomasgard, A., Barton, P.I.: Nonconvex generalized Benders decomposition for stochastic separable mixed-integer nonlinear programs. J. Optim. Theory Appl. 151, 425–454 (2011)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Liberti, L., Pantelides, C.C.: Convex envelopes of monomials of odd degree. J. Glob. Optim. 25, 157–168 (2003)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Lin, Y., Schrage, L.: The global solver in the LINDO API. Optim. Methods Softw. 24, 657–668 (2009)MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Mäkelä, M.M.: Multiobjective proximal bundle method for nonconvex nonsmooth optimization: Fortran subroutine MPBNGC 2.0. Reports of the Department of Mathematical Information Technology, Series B, Scientific computing B 13/2003, University of Jyväskylä (2003)Google Scholar
  36. 36.
    Maly, T., Petzold, L.R.: Numerical methods and software for sensitivity analysis of differential-algebraic systems. Appl. Numer. Math. 20, 57–79 (1996)MathSciNetCrossRefMATHGoogle Scholar
  37. 37.
    Mangasarian, O.L.: A simple characterization of solution sets of convex programs. Oper. Res. Lett. 7(1), 21–26 (1988)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    McCormick, G.P.: Computability of global solutions to factorable nonconvex programs: Part I—convex underestimating problems. Math. Program. 10, 147–175 (1976)CrossRefMATHGoogle Scholar
  39. 39.
    Misener, R., Floudas, C.A.: ANTIGONE: algorithms for continuous/integer global optimization of nonlinear equations. J. Glob. Optim. 59, 503–526 (2014)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Mitsos, A., Chachuat, B., Barton, P.I.: McCormick-based relaxations of algorithms. SIAM J. Optim. 20, 573–601 (2009)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Moore, R.E.: Methods and Applications of Interval Analysis. SIAM, Philadelphia (1979)CrossRefMATHGoogle Scholar
  42. 42.
    Najman, J., Mitsos, A.: Convergence analysis of multivariate McCormick relaxations. J. Glob. Optim. in press (2016)Google Scholar
  43. 43.
    Naumann, U.: The Art of Differentiating Computer Programs. SIAM, Philadelphia (2012)MATHGoogle Scholar
  44. 44.
    Neumaier, A.: Interval Methods for Systems of Equations. Cambridge University Press, Cambridge (1990)MATHGoogle Scholar
  45. 45.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer series in operations research and financial engineering, 2nd edn. Springer, New York (2006)MATHGoogle Scholar
  46. 46.
    Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Classics in Applied Mathematics. SIAM, Philadelphia (2000)CrossRefMATHGoogle Scholar
  47. 47.
    Qi, L., Sun, D.: Smoothing functions and smoothing Newton method for complementarity and variational inequality problems. J. Optim. Theory Appl. 113, 121–147 (2002)MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Rockafellar, R.T.: Convex Analysis. Princeton Landmarks in Mathematics and Physics. Princeton University Press, Princeton (1970)Google Scholar
  49. 49.
    Ryoo, H.S., Sahinidis, N.V.: Global optimization of nonconvex NLPs and MINLPs with applications in process design. Comput. Chem. Eng. 19(5), 551–566 (1995)CrossRefGoogle Scholar
  50. 50.
    Sahinidis, N.V.: BARON: a general purpose global optimization software package. J. Glob. Optim. 8, 201–205 (1996)MathSciNetCrossRefMATHGoogle Scholar
  51. 51.
    Sahinidis, N.V.: BARON 15.9: Global Optimization of Mixed-Integer Nonlinear Programs, User’s Manual. https://www.gams.com/help/topic/gams.doc/solvers/baron/index.html (2015)
  52. 52.
    Schaber, S.D.: Tools for dynamic model development. Ph.D. thesis, Massachusetts Institute of Technology (2014)Google Scholar
  53. 53.
    Scholz, D.: Theoretical rate of convergence for interval inclusion functions. J. Glob. Optim. 53, 749–767 (2012)MathSciNetCrossRefMATHGoogle Scholar
  54. 54.
    Scott, J.K.: Reachability analysis and deterministic global optimization of differential-algebraic systems. Ph.D. thesis, Massachusetts Institute of Technology (2012)Google Scholar
  55. 55.
    Scott, J.K., Barton, P.I.: Convex and concave relaxations for the parametric solutions of semi-explicit index-one differential-algebraic equations. J. Optim. Theory Appl. 156, 617–649 (2013)MathSciNetCrossRefMATHGoogle Scholar
  56. 56.
    Scott, J.K., Barton, P.I.: Improved relaxations for the parametric solutions of ODEs using differential inequalities. J. Glob. Optim. 57, 143–176 (2013)MathSciNetCrossRefMATHGoogle Scholar
  57. 57.
    Scott, J.K., Chachuat, B., Barton, P.I.: Nonlinear convex and concave relaxations for the solutions of parametric ODEs. Optim. Control Appl. Methods 34, 145–163 (2013)MathSciNetCrossRefMATHGoogle Scholar
  58. 58.
    Scott, J.K., Stuber, M.D., Barton, P.I.: Generalized McCormick relaxations. J. Glob. Optim. 51, 569–606 (2011)MathSciNetCrossRefMATHGoogle Scholar
  59. 59.
    Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer series in computational mathematics. Springer, Berlin (1985)CrossRefMATHGoogle Scholar
  60. 60.
    Stuber, M.D., Scott, J.K., Barton, P.I.: Convex and concave relaxations of implicit functions. Optim. Methods Softw. 30(3), 424–460 (2015)MathSciNetCrossRefMATHGoogle Scholar
  61. 61.
    Tawarmalani, M., Sahinidis, N.V.: Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming: Theory, Algorithms, Software, and Applications. Nonconvex Optimization and Its Applications. Springer, Dordrecht (2002)CrossRefMATHGoogle Scholar
  62. 62.
    Tawarmalani, M., Sahinidis, N.V.: Global optimization of mixed-integer nonlinear programs: a theoretical and computational study. Math. Program. A 99, 563–591 (2004)MathSciNetCrossRefMATHGoogle Scholar
  63. 63.
    Tsoukalas, A., Mitsos, A.: Multivariate McCormick relaxations. J. Glob. Optim. 59, 633–662 (2014)MathSciNetCrossRefMATHGoogle Scholar
  64. 64.
    Watson, H.A.J., Khan, K.A., Barton, P.I.: Multistream heat exchanger modeling and design. AIChE J. 61(10), 3390–3403 (2015)CrossRefGoogle Scholar
  65. 65.
    Wechsung, A.: Global optimization in reduced space. Ph.D. thesis, Massachusetts Institute of Technology (2014)Google Scholar
  66. 66.
    Wechsung, A., Aspelund, A., Gundersen, T., Barton, P.I.: Synthesis of heat exchanger networks at subambient conditions with compression and expansion of process streams. AIChE J. 57(8), 2090–2108 (2011)CrossRefGoogle Scholar
  67. 67.
    Wechsung, A., Schaber, S.D., Barton, P.I.: The cluster problem revisited. J. Glob. Optim. 58, 429–438 (2014)MathSciNetCrossRefMATHGoogle Scholar
  68. 68.
    Wechsung, A., Scott, J.K., Watson, H.A.J., Barton, P.I.: Reverse propagation of McCormick relaxations. J. Glob. Optim. 63(1), 1–36 (2015)MathSciNetCrossRefMATHGoogle Scholar
  69. 69.
    Whitney, H.: Analytic extensions of differentiable functions defined in closed sets. Trans. Amer. Math. Soc. 36, 63–89 (1934)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Kamil A. Khan
    • 1
  • Harry A. J. Watson
    • 2
  • Paul I. Barton
    • 2
  1. 1.Mathematics and Computer Science DivisionArgonne National LaboratoryLemontUSA
  2. 2.Process Systems Engineering LaboratoryMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations