Abstract
Deterministic nonconvex optimization solvers generate convex relaxations of nonconvex functions by making use of underlying factorable representations. One approach introduces auxiliary variables assigned to each factor that lifts the problem into a higher-dimensional decision space. In contrast, a generalized McCormick relaxation approach offers the significant advantage of constructing relaxations in the lower dimensionality space of the original problem without introducing auxiliary variables, often referred to as a “reduced-space” approach. Recent contributions illustrated how additional nontrivial inequality constraints may be used in factorable programming to tighten relaxations of the ubiquitous bilinear term. In this work, we exploit an analogous representation of McCormick relaxations and factorable programming to formulate tighter relaxations in the original decision space. We develop the underlying theory to generate necessarily tighter reduced-space McCormick relaxations when a priori convex/concave relaxations are known for intermediate bilinear terms. We then show how these rules can be generalized within a McCormick relaxation scheme via three different approaches: the use of a McCormick relaxations coupled to affine arithmetic, the propagation of affine relaxations implied by subgradients, and an enumerative approach that directly uses relaxations of each factor. The developed approaches are benchmarked on a library of optimization problems using the EAGO.jl optimizer. Two case studies are also considered to demonstrate the developments: an application in advanced manufacturing to optimize supply chain quality metrics and a global dynamic optimization application for rigorous model validation of a kinetic mechanism. The presented subgradient method leads to an improvement in CPU time required to solve the considered problems to \(\epsilon \)-global optimality.
Similar content being viewed by others
Data Availability
The datasets generated during and/or analysed during the current study are available in a GitHub repository: https://github.com/PSORLab/RSBilinear.
References
Anderson, E., Bai, Z., Bischof, C., Blackford, L.S., Demmel, J., Dongarra, J., Croz, J.D., Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia (1999). https://doi.org/10.1137/1.9780898719604
Bao, X., Sahinidis, N.V., Tawarmalani, M.: Multiterm polyhedral relaxations for nonconvex, quadratically constrained quadratic programs. Optim. Methods Softw. 24(4–5), 485–504 (2009). https://doi.org/10.1080/10556780902883184
Bao, X., Sahinidis, N.V., Tawarmalani, M.: Semidefinite relaxations for quadratically constrained quadratic programming: a review and comparisons. Math. Program. 129(1), 129–157 (2011). https://doi.org/10.1007/s10107-011-0462-2
Benson, H.P.: Separable concave minimization via partial outer approximation and branch and bound. Oper. Res. Lett. 9(6), 389–394 (1990). https://doi.org/10.1016/0167-6377(90)90059-E
Bezanson, J., Edelman, A., Karpinski, S., Shah, V.B.: Julia: a fresh approach to numerical computing. SIAM Rev. 59(1), 65–98 (2017). https://doi.org/10.1137/141000671
Blackford, L.S., Demmel, J., Dongarra, J., Duff, I., Hammarling, S., Henry, G., Heroux, M., Kaufman, L., Lumsdaine, A., Petitet, A., Pozo, R., Remington, K., Whaley, R.C.: An updated set of basic linear algebra subprograms (BLAS). ACM Trans. Math. Softw. 28(2), 135–151 (2002). https://doi.org/10.1145/567806.567807
Bompadre, A., Mitsos, A.: Convergence rate of McCormick relaxations. J. Global Optim. 52(1), 1–28 (2011). https://doi.org/10.1007/s10898-011-9685-2
Bongartz, D., Mitsos, A.: Deterministic global optimization of process flowsheets in a reduced space using McCormick relaxations. J. Global Optim. 69(4), 761–796 (2017). https://doi.org/10.1007/s10898-017-0547-4
Bongartz, D., Mitsos, A.: Deterministic global flowsheet optimization: between equation-oriented and sequential-modular methods. AIChE J. 65(3), 1022–1034 (2019). https://doi.org/10.1002/aic.16507
Bongartz, D., Najman, J., Mitsos, A.: Deterministic global optimization of steam cycles using the IAPWS-IF97 model. Optim. Eng. (2020). https://doi.org/10.1007/s11081-020-09502-1
Bongartz, D., Najman, J., Sass, S., Mitsos, A.: MAiNGO: McCormick based algorithm for mixed integer nonlinear global optimization. Technical report, RWTH Aachen (2018). https://git.rwth-aachen.de/avt-svt/public/maingo
Comba, J.L.D., Stolfi, J.: Affine arithmetic and its applications to computer graphics. In: de Figueiredo, L.H., de Miranda Gomes, J. (eds.) Proceedings of VI SIBGRAPI (Brazilian Symposium on Computer Graphics and Image Processing), pp. 9–18 (1993). http://urlib.net/ibi/8JMKD3MGPBW34M/3D6UQ68
de Figueiredo, L.H., Stolfi, J.: Affine arithmetic: concepts and applications. Numer. Algorithms 37, 147–158 (2004). https://doi.org/10.1023/b:numa.0000049462.70970.b6
De Oliveira, L.G., de Paiva, A.P., Balestrassi, P.P., Ferreira, J.R., da Costa, S.C., da Silva Campos, P.H.: Response surface methodology for advanced manufacturing technology optimization: theoretical fundamentals, practical guidelines, and survey literature review. Int. J. Adv. Manuf. Technol. 104(5), 1785–1837 (2019). https://doi.org/10.1007/s00170-019-03809-9
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002). https://doi.org/10.1007/s101070100263
Falk, J.E., Hoffman, K.L.: Concave minimization via collapsing polytopes. Oper. Res. 34(6), 919–929 (1986). https://doi.org/10.1287/opre.34.6.919
Fedorov, G., Nguyen, K.T., Harrison, P., Singh, A.: Intel Math Kernel Library 2019 Update 2 Release Notes (2019). https://software.intel.com/en-us/mkl
Gurobi Optimization, LLC: Gurobi optimizer reference manual (2020). http://www.gurobi.com
He, T., Tawarmalani, M.: A new framework to relax composite functions in nonlinear programs. Mathematical Programming, pp. 1–40 (2020). https://doi.org/10.1007/s10107-020-01541-x
Hejazi, T.H., Seyyed-Esfahani, M., Mahootchi, M.: Quality chain design and optimization by multiple response surface methodology. Int. J. Adv. Manuf. Technol. 68(1–4), 881–893 (2013). https://doi.org/10.1007/s00170-013-4950-9
Hiriart-Urruty, J.B., Lemaréchal, C.: Fundamentals of Convex Analysis. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-642-56468-0
Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches, 3rd edn. Springer, Berlin (1996). https://doi.org/10.1007/978-3-662-03199-5
IBM: IBM ILOG CPLEX Optimizer. https://www.ibm.com/software/integration/optimization/cplex-optimizer/ (2020)
Kannan, R., Barton, P.I.: The cluster problem in constrained global optimization. J. Global Optim. 69(3), 629–676 (2017). https://doi.org/10.1007/s10898-017-0531-z
Khan, K.A., Watson, H.A.J., Barton, P.I.: Differentiable McCormick relaxations. J. Global Optim. 67(4), 687–729 (2016). https://doi.org/10.1007/s10898-016-0440-6
Khan, K.A., Wilhelm, M., Stuber, M.D., Cao, H., Watson, H.A.J., Barton, P.I.: Corrections to: Differentiable McCormick relaxations. J. Global Optim. 70(3), 705–706 (2018). https://doi.org/10.1007/s10898-017-0601-2
McCormick, G.P.: Computability of global solutions to factorable nonconvex programs: Part I—convex underestimating problems. Math. Program. 10(1), 147–175 (1976). https://doi.org/10.1007/bf01580665
Messine, F.: Extensions of affine arithmetic: application to unconstrained global optimization. J. Univ. Comput. Sci. 8(11), 992–1015 (2002). https://doi.org/10.3217/jucs-008-11-0992
Misener, R., Floudas, C.A.: GloMIQO: global mixed-integer quadratic optimizer. J. Global Optim. 57(1), 3–50 (2013). https://doi.org/10.1007/s10898-012-9874-7
Misener, R., Floudas, C.A.: ANTIGONE: algorithms for continuous/integer global optimization of nonlinear equations. J. Global Optim. 59(2–3), 503–526 (2014). https://doi.org/10.1007/s10898-014-0166-2
Mistry, M., Misener, R.: Optimising heat exchanger network synthesis using convexity properties of the logarithmic mean temperature difference. Comput. Chem. Eng. 94, 1–17 (2016). https://doi.org/10.1016/j.compchemeng.2016.07.001
Mitsos, A., Chachuat, B., Barton, P.I.: McCormick-based relaxations of algorithms. SIAM J. Optim. 20(2), 573–601 (2009). https://doi.org/10.1137/080717341
Moore, R.E.: Interval Analysis. Prentice-Hall, Englewood Cliffs, NJ (1966)
Mosek, A.: The MOSEK optimization software (2010). https://www.mosek.com
Mostaan, H., Shamanian, M., Safari, M.: Process analysis and optimization for fracture stress of electron beam welded ultra-thin FeCo-V foils. Int. J. Adv. Manuf. Technol. 87(1), 1045–1056 (2016). https://doi.org/10.1007/s00170-016-8553-0
Nagarajan, H., Lu, M., Wang, S., Bent, R., Sundar, K.: An adaptive, multivariate partitioning algorithm for global optimization of nonconvex programs. J. Global Optim. (2019). https://doi.org/10.1007/s10898-018-00734-1
Nagarajan, H., Lu, M., Yamangil, E., Bent, R.: Tightening McCormick relaxations for nonlinear programs via dynamic multivariate partitioning. In: Rueher, M. (ed.) International Conference on Principles and Practice of Constraint Programming, pp. 369–387. Springer, New York (2016). https://doi.org/10.1007/978-3-319-44953-1_24
Najman, J., Bongartz, D., Mitsos, A.: Convex relaxations of componentwise convex functions. Comput. Chem. Eng. 130, 106527 (2019). https://doi.org/10.1016/j.compchemeng.2019.106527
Najman, J., Bongartz, D., Mitsos, A.: Relaxations of thermodynamic property and costing models in process engineering. Comput. Chem. Eng. 130, 106571 (2019). https://doi.org/10.1016/j.compchemeng.2019.106571
Najman, J., Bongartz, D., Tsoukalas, A., Mitsos, A.: Erratum to: Multivariate McCormick relaxations. J. Global Optim. 68, 219–225 (2017). https://doi.org/10.1007/s10898-016-0470-0
Najman, J., Mitsos, A.: Convergence analysis of multivariate McCormick relaxations. J. Global Optim. 66(4), 1–32 (2016). https://doi.org/10.1007/s10898-016-0408-6
Najman, J., Mitsos, A.: Convergence order of McCormick relaxations of LMTD function in heat exchanger networks. Comput. Aided Chem. Eng. 38, 1605–1610 (2016). https://doi.org/10.1016/B978-0-444-63428-3.50272-1
Najman, J., Mitsos, A.: On tightness and anchoring of McCormick and other relaxations. J. Global Optim. (2017). https://doi.org/10.1007/s10898-017-0598-6
Najman, J., Mitsos, A.: Tighter McCormick relaxations through subgradient propagation. J. Global Optim. 75, 565–593 (2019). https://doi.org/10.1007/s10898-019-00791-0
Nedialkov, N.S., Kreinovich, V., Starks, S.A.: Interval arithmetic, affine arithmetic, Taylor series methods: why, what next? Numer. Algorithms 37(1), 325–336 (2004). https://doi.org/10.1023/B:NUMA.0000049478.42605.cf
Ninin, J., Messine, F., Hansen, P.: A reliable affine relaxation method for global optimization. 4OR 13(3), 247–277 (2015). https://doi.org/10.1007/s10288-014-0269-0
Nohra, C.J., Raghunathan, A.U., Sahinidis, N.: Spectral relaxations and branching strategies for global optimization of mixed-integer quadratic programs. SIAM J. Optim. 31(1), 142–171 (2021). https://doi.org/10.1137/19M1271762
Nohra, C.J., Raghunathan, A.U., Sahinidis, N.V.: SDP-quality bounds via convex quadratic relaxations for global optimization of mixed-integer quadratic programs. Math. Program. (2021). https://doi.org/10.1007/s10107-021-01680-9
Pakseresht, A.H., Ghasali, E., Nejati, M., Shirvanimoghaddam, K., Javadi, A.H., Teimouri, R.: Development empirical-intelligent relationship between plasma spray parameters and coating performance of Yttria–Stabilized zirconia. Int. J. Adv. Manuf. Technol. 76(5), 1031–1045 (2015). https://doi.org/10.1007/s00170-014-6212-x
Puranik, Y., Sahinidis, N.V.: Domain reduction techniques for global NLP and MINLP optimization. Constraints 22(3), 338–376 (2017). https://doi.org/10.1007/s10601-016-9267-5
Rajakumar, S., Balasubramanian, V.: Diffusion bonding of titanium and aa 7075 aluminum alloy dissimilar joints-process modeling and optimization using desirability approach. Int. J. Adv. Manuf. Technol. 86(1), 1095–1112 (2016). https://doi.org/10.1007/s00170-015-8223-7
Rajesh, P., Nagaraju, U., Gowd, G.H., Vardhan, T.V.: Experimental and parametric studies of ND: Yag laser drilling on austenitic stainless steel. Int. J. Adv. Manuf. Technol. 93(1), 65–71 (2017). https://doi.org/10.1007/s00170-015-7639-4
Rump, S.M., Kashiwagi, M.: Implementation and improvements of affine arithmetic. Nonlinear Theory Appl. IEICE 6(3), 341–359 (2015). https://doi.org/10.1587/nolta.6.341
Sahinidis, N.V.: BARON: A general purpose global optimization software package. J. Global Optim. 8(2), 201–205 (1996). https://doi.org/10.1007/BF00138693
Sahinidis, N.V.: BARON 21.1.13: Global Optimization of Mixed-Integer Nonlinear Programs. User’s Manual (2017). https://www.minlp.com/downloads/docs/baron%20manual.pdf
Sahlodin, A.M., Chachuat, B.: Discretize-then-relax approach for convex/concave relaxations of the solutions of parametric ODEs. Appl. Numer. Math. 61(7), 803–820 (2011). https://doi.org/10.1016/j.apnum.2011.01.009
Schweidtmann, A.M., Bongartz, D., Grothe, D., Kerkenhoff, T., Lin, X., Najman, J., Mitsos, A.: Deterministic global optimization with gaussian processes embedded. Math. Program. Comput. 13(3), 553–581 (2021). https://doi.org/10.1007/s12532-021-00204-y
Schweidtmann, A.M., Bongartz, D., Huster, W.R., Mitsos, A.: Deterministic global process optimization: Flash calculations via artificial neural networks. In: Kiss, A.A., Zondervan, E., Lakerveld, R., Özkan, L. (eds.) Computer Aided Chemical Engineering, vol. 46, pp. 937–942. Elsevier, Amsterdam (2019). https://doi.org/10.1016/b978-0-12-818634-3.50157-0
Schweidtmann, A.M., Huster, W.R., Lüthje, J.T., Mitsos, A.: Deterministic global process optimization: accurate (single-species) properties via artificial neural networks. Comput. Chem. Eng. 121, 67–74 (2019). https://doi.org/10.1016/j.compchemeng.2018.10.007
Schweidtmann, A.M., Mitsos, A.: Deterministic global optimization with artificial neural networks embedded. J. Optim. Theory Appl. 180(3), 925–948 (2018). https://doi.org/10.1007/s10957-018-1396-0
Scott, J.K., Barton, P.I.: Improved relaxations for the parametric solutions of odes using differential inequalities. J. Global Optim. 57(1), 143–176 (2013). https://doi.org/10.1007/s10898-012-9909-0
Scott, J.K., Stuber, M.D., Barton, P.I.: Generalized McCormick relaxations. J. Global Optim. 51(4), 569–606 (2011). https://doi.org/10.1007/s10898-011-9664-7
Singer, A.B.: Global dynamic optimization. Ph.D. thesis, Massachusetts Institute of Technology (2004)
Stolfi, J., Figueiredo, L.H.D.: An introduction to affine arithmetic. TEMA Tendências em Matemática Aplicada e Computacional 4(3), 297–312 (2003). https://doi.org/10.5540/tema.2003.04.03.0297
Stuber, M.D., Scott, J.K., Barton, P.I.: Convex and concave relaxations of implicit functions. Optim. Methods Softw. 30(3), 424–460 (2015). https://doi.org/10.1080/10556788.2014.924514
Tawarmalani, M., Sahinidis, N.V.: A polyhedral branch-and-cut approach to global optimization. Math. Program. 103(2), 225–249 (2005). https://doi.org/10.1007/s10107-005-0581-8
Tsoukalas, A., Mitsos, A.: Multivariate McCormick relaxations. J. Global Optim. 59(2–3), 633–662 (2014). https://doi.org/10.1007/s10898-014-0176-0
Wang, C., Wilhelm, M.E., Stuber, M.D.: Semi-Infinite optimization with hybrid models. Ind. Eng. Chem. Res. 61(15), 5239–5254 (2022). https://doi.org/10.1021/acs.iecr.2c00113
Wang, E., Zhang, Q., Shen, B., Zhang, G., Lu, X., Wu, Q., Wang, Y.: Intel Math Kernel Library, p. 167. Springer, New York (2014). https://doi.org/10.1007/978-3-319-06486-4_7
Watson, H.A., Vikse, M., Gundersen, T., Barton, P.I.: Optimization of single mixed-refrigerant natural gas liquefaction processes described by nondifferentiable models. Energy 150, 860–876 (2018). https://doi.org/10.1016/j.energy.2018.03.013
Wechsung, A., Scott, J.K., Watson, H.A.J., Barton, P.I.: Reverse propagation of McCormick relaxations. J. Global Optim. 63(1), 1–36 (2015). https://doi.org/10.1007/s10898-015-0303-6
Wilhelm, M.E., Gottlieb, R.X., Stuber, M.D.: PSORLab/McCormick.jl (2020). https://doi.org/10.5281/ZENODO.4278415. https://github.com/PSORLab/McCormick.jl
Wilhelm, M.E., Le, A.V., Stuber, M.D.: Global optimization of stiff dynamical systems. AIChE J. 65, 16836 (2019). https://doi.org/10.1002/aic.16836
Wilhelm, M.E., Stuber, M.D.: EAGO.jl easy advanced global optimization in Julia. Optim. Methods Softw. 1, 23 (2020). https://doi.org/10.1080/10556788.2020.1786566
Wilhelm, M.E., Wang, C., Stuber, M.D.: Convex and concave envelopes of artificial neural network activation functions for deterministic global optimization. J. Glob. Optim. (2022). https://doi.org/10.1007/s10898-022-01228-x
Yue, Z., Huang, C., Zhu, H., Wang, J., Yao, P., Liu, Z.: Optimization of machining parameters in the abrasive waterjet turning of alumina ceramic based on the response surface methodology. Int. J. Adv. Manuf. Technol. 71(9–12), 2107–2114 (2014). https://doi.org/10.1007/s00170-014-5624-y
Zorn, K., Sahinidis, N.V.: Global optimization of general non-convex problems with intermediate bilinear substructures. Optim. Methods Softw. 29(3), 442–462 (2014). https://doi.org/10.1080/10556788.2013.783032
Acknowledgements
This material is based upon work supported by the National Science Foundation under Grant No. 1932723. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We also gratefully acknowledge the Air Force Research Laboratory, Materials and Manufacturing Directorate (AFRL/RXMS) for support via Contract No. FA8650-20-C-5206. The views, opinions, and/or findings contained in this paper are those of the authors and should not be interpreted as representing the official views or policies, either expressed or implied, of the Air Force Research Laboratory, the United States Air Force, or the Department of Defense.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Communicated by Alexander Mitsos.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wilhelm, M.E., Stuber, M.D. Improved Convex and Concave Relaxations of Composite Bilinear Forms. J Optim Theory Appl 197, 174–204 (2023). https://doi.org/10.1007/s10957-023-02196-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-023-02196-2
Keywords
- Deterministic global optimization
- Nonconvex programming
- McCormick relaxations
- Branch-and-bound
- Multilinear products