Alternative regularizations for Outer-Approximation algorithms for convex MINLP

Abstract

In this work, we extend the regularization framework from Kronqvist et al. (Math Program 180(1):285–310, 2020) by incorporating several new regularization functions and develop a regularized single-tree search method for solving convex mixed-integer nonlinear programming (MINLP) problems. We propose a set of regularization functions based on distance metrics and Lagrangean approximations, used in the projection problem for finding new integer combinations to be used within the Outer-Approximation (OA) method. The new approach, called Regularized Outer-Approximation (ROA), has been implemented as part of the open-source Mixed-integer nonlinear decomposition toolbox for Pyomo—MindtPy. We compare the OA method with seven regularization function alternatives for ROA. Moreover, we extend the LP/NLP Branch and Bound method proposed by Quesada and Grossmann (Comput Chem Eng 16(10–11):937–947, 1992) to include regularization in an algorithm denoted RLP/NLP. We provide convergence guarantees for both ROA and RLP/NLP. Finally, we perform an extensive computational experiment considering all convex MINLP problems in the benchmark library MINLPLib. The computational results show clear advantages of using regularization combined with the OA method.

Appendix

1.1 Algorithmic description of OA and LP/NLP branch and bound

This section presents the algorithmic description of the Outer-Approximation method [15, 17], in Algorithm 3, and the LP/NLP Branch and Bound method [6, 44], in Algorithm 4.

1.2 Representing $\ell _1$ and $\ell _\infty$ norms using linear programming

This section shows the valid reformulations of optimization problems with norms one and infinity in the objective function using auxiliary variables and linear constraints. This reformulation is exact in the sense that they preserve the local and global optima from the original problem [39]. These reformulations are particularly interesting since they allow the regularization problem MIP-Proj to be written as Mixed-Integer Linear Programming (MILP) problems, instead of Mixed-Integer Quadratic Programming (MIQP) problems, as in the work by Kronqvist et al. [35]. MILP solution methods’ maturity over MIQP allows these problems to be more quickly solvable in practice.

The norm-1 of a vector ${\mathbf {v}} \in V \subseteq {\mathbb {R}}^N$ whose components might be negative or positive, $\ell _1({\mathbf {v}})=\left\| {\mathbf {v}}\right\| _1 = \sum _{i=1}^{N} |v_i|$ can be reformulated in the case that this term appears in the objective function with a set of linear constraints. Through the addition of 2N non-negative slack variables ${\mathbf {s}}^+,{\mathbf {s}}^- \in {\mathbb {R}}_+^{N}$, and N linear equality constraints the following reformulation is valid:

$$\begin{aligned} \begin{array}{ll} \min \limits _{{\mathbf {v}}} &{}\quad \left\| {\mathbf {v}}\right\| _1 \\ \text {s.t.}\, &{}\quad {\mathbf {v}} \in V \subseteq {\mathbb {R}}^N \end{array} \Leftrightarrow \begin{array}{ll} \min \limits _{{\mathbf {v}}, {\mathbf {s}}^+, {\mathbf {s}}^-} &{}\quad \sum _{i=1}^{N} s_i^+ + s_i^- \\ \text {s.t.}\, &{}\quad {\mathbf {s}}^+ - {\mathbf {s}}^- = {\mathbf {v}}\\ &{}\quad {\mathbf {v}} \in V \subseteq {\mathbb {R}}^N,\quad {\mathbf {s}}^+ \in {\mathbb {R}}_+^{N},\quad {\mathbf {s}}^- \in {\mathbb {R}}_+^{N}. \end{array} \end{aligned}$$

(18)

This reformulation is applied to the regularization problem MIP-Proj when considering the $\ell _1$ regularization function as in (4), resulting in problem MIP-Proj-$\ell _1$. It can also be potentially applied to the feasibility NLP problem NLP-f.

Fig. 11

Time performance profile for multi-tree ROA method as described in Algorithm 1. (Color figure online)

Fig. 12

Iteration performance profile for multi-tree ROA method as described in Algorithm 1. (Color figure online)

Fig. 13

Time performance profile for single-tree RLP/NLP methods as described in Algorithm 2. (Color figure online)

Fig. 14

Iteration performance profile single-tree RLP/NLP methods as described in Algorithm 2. (Color figure online)

The norm-$\infty$ of a vector ${\mathbf {v}} \in V \subseteq {\mathbb {R}}^N$ whose components might be negative or positive, $\ell _\infty ({\mathbf {v}})=\left\| {\mathbf {v}}\right\| _\infty = \max _{i=\{1,\ldots ,N\}} |v_i|$ can be reformulated in the case that this term appears in the objective function with a set of linear constraints. Through the addition of one non-negative slack variable $s \in {\mathbb {R}}_+$, and 2N linear inequality constraints, the following reformulation is valid:

$$\begin{aligned} \begin{array}{ll} \min \limits _{{\mathbf {v}}} &{}\quad \left\| {\mathbf {v}}\right\| _\infty \\ \text {s.t.}\, &{}\quad {\mathbf {v}} \in V \subseteq {\mathbb {R}}^N \end{array} \Leftrightarrow \begin{array}{ll} \min \limits _{{\mathbf {v}}, s} &{}\quad s \\ \text {s.t.}\, &{}\quad s \ge {\mathbf {v}}\\ &{}\quad s \ge -{\mathbf {v}}\\ &{}\quad {\mathbf {v}} \in V \subseteq {\mathbb {R}}^N,\quad s \in {\mathbb {R}}_+ . \end{array} \end{aligned}$$

(19)

This is the usual choice for reformulating problem NLP-f, and can also be used to reformulate problem MIP-Proj with $\ell _\infty$ regularization objective function, as in (5). This last problem formulation is:

1.3 Performance profiles for problem set 1

This section of the Appendix presents the performance profiles for the multi-tree and single-tree implementation of the methods included in this manuscript when solving all 358 convex MINLP problems in Problem Set 1. Figures 11 and 12 include the time and iteration performance profiles for the multi-tree implementation, respectively. Figures 13 and 14 include the time and iteration performance profiles for the single-tree implementation, respectively. Notice that we define iterations in the single-tree context as the number of NLP-I problems solved.