Skip to main content
SpringerLink
Log in

General convex relaxations of implicit functions and inverse functions

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

Convex relaxations of nonconvex functions provide useful bounding information in applications such as deterministic global optimization and reachability analysis. In some situations, the original nonconvex functions may not be known explicitly, but are instead described implicitly by nonlinear equation systems. In these cases, established convex relaxation methods for closed-form functions are not directly applicable. This article presents a new general strategy to construct convex relaxations for such implicit functions. These relaxations are described as convex parametric programs whose constraints are convex relaxations of the original residual function. This relaxation strategy is straightforward to implement, produces tight relaxations in practice, is particularly efficient to carry out when monotonicity properties can be exploited, and does not assume the existence or uniqueness of an implicit function on the entire intended domain. Unlike all previous approaches to the authors’ knowledge, this new approach permits any relaxations of the residual function; it does not require the residual relaxations to be factorable or to be obtained from a McCormick-like traversal of a computational graph. This new convex relaxation strategy is extended to inverse functions, compositions involving implicit functions, feasible-set mappings in constraint satisfaction problems, and solutions of parametric ODEs. Based on a proof-of-concept implementation in Julia, numerical examples are presented to illustrate the convex relaxations produced for various implicit functions and optimal-value functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availibility statement

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Code Availability

Our Julia code for our numerical examples is available at https://github.com/kamilkhanlab/implicit-func-relaxations

References

  1. Mitsos, A., Chachuat, B., Barton, P.I.: McCormick-based relaxations of algorithms. SIAM J. Optim. 20(2), 573–601 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Adjiman, C.S., Dallwig, S., Floudas, C.A., Neumaier, A.: A global optimization method, \(\alpha \)BB, for general twice-differentiable constrained NLPs - I. Theor. Adv. Comput. Chem. Eng. 22(9), 1137–1158 (1998)

    Article  Google Scholar 

  3. McCormick, G.P.: Computability of global solutions to factorable nonconvex programs: part I-convex underestimating problems. Math. Program. 10(1), 147–175 (1976)

    Article  MATH  Google Scholar 

  4. Khan, K.A., Watson, H.A.J., Barton, P.I.: Differentiable McCormick relaxations. J. Global Optim. 67(4), 687–729 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chachuat, B.: MC++: a toolkit for bounding factorable functions (2014)

  6. Wilhelm, M.E., Stuber, M.D.: EAGO.jl: Easy Advanced Global Optimization in Julia. Optim. Methods Softw. 1–26 (2020)

  7. Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Mappings: A View from Variational Analysis, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer, New York (2014)

    Book  MATH  Google Scholar 

  8. Wechsung, A., Scott, J.K., Watson, H.A.J., Barton, P.I.: Reverse propagation of McCormick relaxations. J. Global Optim. 63(1), 1–36 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  9. Scott, J.K., Stuber, M.D., Barton, P.I.: Generalized McCormick relaxations. J. Global Optim. 51(4), 569–606 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  10. Stuber, M.D., Scott, J.K., Barton, P.I.: Convex and concave relaxations of implicit functions. Optim. Methods Softw. 30(3), 424–460 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, New York (2000)

  12. Khan, K.A., Wilhelm, M., Stuber, M.D., Cao, H., Watson, H.A.J., Barton, P.I.: Corrections to: differentiable McCormick relaxations. J. Global Optim. 70(3), 705–706 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. Wilhelm, M.E., Le, A.V., Stuber, M.D.: Global optimization of stiff dynamical systems. AIChE J. 65(12), e16836 (2019)

    Article  Google Scholar 

  14. Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA (2008)

    Book  MATH  Google Scholar 

  15. Tsoukalas, A., Mitsos, A.: Multivariate McCormick relaxations. J. Global Optim. 59(2–3), 633–662 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  16. Smith, E.M.B., Pantelides, C.C.: Global optimisation of nonconvex MINLPs. Comput. Chem. Eng. 21, 791–796 (1997)

    Article  Google Scholar 

  17. Scott, J.K., Barton, P.I.: Improved relaxations for the parametric solutions of ODEs using differential inequalities. J. Global Optim. 57(1), 143–176 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  18. Song, Y., Cao, H., Mehta, C., Khan, K.A.: Bounding convex relaxations of process models from below by tractable black-box sampling. Comput. Chem. Eng. 153, 107413 (2021)

    Article  Google Scholar 

  19. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I: Fundamentals. Springer, Berlin (1993)

    Book  MATH  Google Scholar 

  20. Cao, H., Khan, K.A.: Enclosing reachable sets for nonlinear control systems using complementarity-based intervals. IFAC-PapersOnLine 54(3), 590–595 (2021)

    Article  Google Scholar 

  21. Fiacco, A.V., Kyparisis, J.: Convexity and concavity properties of the optimal value function in parametric nonlinear programming. J. Optim. Theory Appl. 48(1), 95–126 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  22. Scholtes, S.: Introduction to Piecewise Differentiable Equations. SpringerBriefs in Optimization. Springer, New York (2012)

    Book  MATH  Google Scholar 

  23. Hansen, E., Greenberg, R.: An interval Newton method. Appl. Math. Comput. 12(2–3), 89–98 (1983)

    MathSciNet  MATH  Google Scholar 

  24. Neumaier, A.: Interval Methods for Systems of Equations. Encyclopedia of Mathematics and Its Applications. Cambridge University Press, Cambridge (1991)

  25. Clarke, F.: Optimization and Nonsmooth Analysis. Classics in Applied Mathematics. SIAM, Philadelphia (1990)

  26. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  27. Drud, A.S.: CONOPT - a large-scale GRG code. ORSA J. Comput. 6(2), 207–216 (1994)

    Article  MATH  Google Scholar 

  28. Nesterov, Y.: Lectures on Convex Optimization. Springer Optimization and Its Applications. Springer, Cham (2018)

  29. Beckers, M., Mosenkis, V., Naumann, U.: Adjoint mode computation of subgradients for McCormick relaxations. In: Forth, S., Hovland, P., Phipps, E., Utke, J., Walther, A. (eds.) Recent Advances in Algorithmic Differentiation, pp. 103–113. Springer, Berlin (2012)

    Chapter  Google Scholar 

  30. Khan, K.A.: Branch-locking AD techniques for nonsmooth composite functions and nonsmooth implicit functions. Optim. Methods Softw. 33(4–6), 1127–1155 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  31. Hansen, E., Sengupta, S.: Bounding solutions of systems of equations using interval analysis. BIT 21(2), 203–211 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  32. Bompadre, A., Mitsos, A.: Convergence rate of McCormick relaxations. J. Global Optim. 52(1), 1–28 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  33. Nerantzis, D., Adjiman, C.S.: Tighter \(\alpha \)BB relaxations through a refinement scheme for the scaled Gerschgorin theorem. J. Global Optim. 73(3), 467–483 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  34. Cao, H., Khan, K.A.: Bounding nonconvex optimal control problems using Pontryagin’s minimum principle (Submitted) (2021)

  35. Song, Y., Khan, K.A.: Optimization-based convex relaxations for nonconvex parametric systems of ordinary differential equations. Math. Program. 196, 521–565 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  36. Ralph, D., Dempe, S.: Directional derivatives of the solution of a parametric nonlinear program. Math. Program. 70(1–3), 159–172 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  37. Khan, K.A., Yuan, Y.: Constructing a subgradient from directional derivatives for functions of two variables. J. Nonsmooth Anal. Optim. 6061 (2020)

  38. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton, NJ (1996)

    Google Scholar 

  39. Gleixner, A.M., Berthold, T., Müller, B., Weltge, S.: Three enhancements for optimization-based bound tightening. J. Global Optim. 67(4), 731–757 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  40. Zamora, J.M., Grossmann, I.E.: A branch and contract algorithm for problems with concave univariate, bilinear and linear fractional terms. J. Global Optim. 14(3), 217–249 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  41. Misener, R., Floudas, C.A.: ANTIGONE: Algorithms for coNTinuous/Integer Global Optimization of Nonlinear Equations. J. Global Optim. 59(2–3), 503–526 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Bezanson, J., Edelman, A., Karpinski, S., Shah, V.: Julia: a fresh approach to numerical computing. SIAM Rev. 59(1), 65–98 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  43. Dunning, I., Huchette, J., Lubin, M.: JuMP: a modeling language for mathematical optimization. SIAM Rev. 59(2), 295–320 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  44. Atkins, P., de Paula, J.: Physical Chemistry, 8th edn. W. H. Freeman, New York (2006)

    Google Scholar 

  45. Ascher, U.M., Petzold, L.R.: Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations, 1st edn. Society for Industrial and Applied Mathematics, USA (1998)

    Book  MATH  Google Scholar 

  46. Sahlodin, A.M., Chachuat, B.: Convex/concave relaxations of parametric ODEs using Taylor models. Comput. Chem. Eng. 35(5), 844–857 (2011)

    Article  MATH  Google Scholar 

  47. Hartman, P.: Ordinary Differential Equations. SIAM, Philadelphia, PA (2002)

    Book  MATH  Google Scholar 

  48. Sahlodin, A.M., Chachuat, B.: Discretize-then-relax approach for convex/concave relaxations of the solutions of parametric ODEs. Appl. Numer. Math. 61(7), 803–820 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

This work was supported by the McMaster Advanced Control Consortium (MACC), and by the Natural Sciences and Engineering Research Council of Canada (NSERC) under Grant RGPIN-2017-05944.

Author information

Authors and Affiliations

  1. Department of Chemical Engineering, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4L8, Canada

    Huiyi Cao & Kamil A. Khan

Authors
  1. Huiyi Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamil A. Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design, and to the formulation and proofs of mathematical results. Numerical experiments were performed by Huiyi Cao. All authors wrote, edited, read, and approved the final manuscript.

Corresponding author

Correspondence to Kamil A. Khan.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, H., Khan, K.A. General convex relaxations of implicit functions and inverse functions. J Glob Optim 86, 545–572 (2023). https://doi.org/10.1007/s10898-023-01281-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-023-01281-0

Keywords

Search

Navigation