Compact mixed-integer programming formulations in quadratic optimization

Abstract

We present a technique for producing valid dual bounds for nonconvex quadratic optimization problems. The approach leverages an elegant piecewise linear approximation for univariate quadratic functions due to Yarotsky (Neural Netw 94:103–114, 2017), formulating this (simple) approximation using mixed-integer programming (MIP). Notably, the number of constraints, binary variables, and auxiliary continuous variables used in this formulation grows logarithmically in the approximation error. Combining this with a diagonal perturbation technique to convert a nonseparable quadratic function into a separable one, we present a mixed-integer convex quadratic relaxation for nonconvex quadratic optimization problems. We study the strength (or sharpness) of our formulation and the tightness of its approximation. Further, we show that our formulation represents feasible points via a Gray code. We close with computational results on problems with quadratic objectives and/or constraints, showing that our proposed method (i) across the board outperforms existing MIP relaxations from the literature, and (ii) on hard instances produces better bounds than exact solvers within a fixed time budget.

Appendices

Normalized multi-parametric disaggregation technique

We present a standard approach to descretizing continuous variables for handling bilinear products in nonlinear models. This approach is perhaps the most straightforward way to convert bilinear problems to MILPs and has been referred to as Normalized Multi-Parametric Disaggregation Technique (NMDT) [14]. We adapt the bilinear approach here to a squaring a single variable.

Consider $x \in [0,1]$, and let L be a positive integer. We then use the representation

$$\begin{aligned} x&= \sum _{i=1}^p 2^{-i} \beta _i + \varDelta x \end{aligned}$$

(52a)

$$\begin{aligned} \beta _i&\in \{0,1\}&i \in \llbracket L \rrbracket \end{aligned}$$

(52b)

$$\begin{aligned} \varDelta x&\in [0, 2^{-L}], \end{aligned}$$

(52c)

where L is the number of binary variables to use.

Multiplying (52a) by x, and substituting the representation into the $x \varDelta x$ term, we obtain

$$\begin{aligned} y&= x \cdot x = \sum _{i=1}^L 2^{-i} x\beta _i + x\varDelta x\\&= \sum _{i=1}^L 2^{-i} x\beta _i + \left( \sum _{i=1}^L 2^{-i} \beta _i + \varDelta x\right) \varDelta x \\&= \sum _{i=1}^L 2^{-i} (x + \varDelta x) \beta _i + \varDelta x^2 \end{aligned}$$

Now, using the fact that $x + \varDelta x \in [0, 1+2^{-L}]$, first lift the model by adding variables $u_i$ and $\varDelta u$ such that $u_i = (x + \varDelta x) \beta _i$ and $\varDelta u = \varDelta x^2$, and then we relax these equations using McCormick Envelopes.

Given bounds $x \in [ {x}^{\min }, {x}^{\max }]$ and $\beta \in [0, 1]$, The McCormick envelope ${\mathcal {M}}(x,\beta )$ is defined as the following relaxation of $u = x \beta$

$$\begin{aligned}&{\mathcal {M}}(x, \beta ) = \left\{ (x, \beta , y) \in {[} {x}^{\min }, {x}^{\max }]\times [0,1] \times {\mathbb {R}}\ : (55) \right\} . \end{aligned}$$

(54)

$$\begin{aligned}&{x}^{\min } \cdot \beta \le \, u \le {x}^{\max }\cdot \beta \nonumber \\&\quad x - {x}^{\max }\cdot (1-\beta ) \le \, u \le x - {x}^{\min }\cdot (1-\beta ) \end{aligned}$$

(55)

To approximate $u = x^2$ with $x \in [0, {x}^{\max }]$, this becomes

$$\begin{aligned} {\mathcal {M}}(x) = \left\{ (x, u) \in [0, {x}^{\max }] \times {\mathbb {R}}\ : \ \begin{aligned}&\, u \ge 0 \\ {x}^{\max } (2 x - {x}^{\max }) \le&\, u \le {x}^{\max }\cdot x \end{aligned} \right\} . \end{aligned}$$

(56)

We present two ways to use this approach. The first is the most direct use of $\texttt {NMDT}{}$, as used in [14]. This model is

$$\begin{aligned} x&= \sum _{i=1}^L 2^{-i} \beta _i + \varDelta x\end{aligned}$$

(57a)

$$\begin{aligned} y&= \sum _{i=1}^L 2^{-i} u_i + \varDelta u\end{aligned}$$

(57b)

$$\begin{aligned} (x, \beta _i, u_i)&\in {\mathcal {M}}(x, \beta _i) \quad i \in \llbracket L \rrbracket \end{aligned}$$

(57c)

$$\begin{aligned} (\varDelta x, x, \varDelta u)&\in {\mathcal {M}}(\varDelta x, x)\end{aligned}$$

(57d)

$$\begin{aligned} \beta _i&\in \{0,1\} \quad i \in \llbracket L \rrbracket \end{aligned}$$

(57e)

$$\begin{aligned} \varDelta x&\in [0, 2^{-L}] \end{aligned}$$

(57f)

Here, the only error introduced in the relaxation is from $\varDelta u = x\varDelta x$, yielding a maximum error of $2^{-L-2}$, again occurring when $\varDelta x = 2^{-L-1}$.

Alternatively, we consider the expansion of the $x \varDelta x$ term. We thus obtain the T-NMDT relaxation for $y=x^2$.

$$\begin{aligned} x&= \sum _{i=1}^L 2^{-i} \beta _i + \varDelta x\end{aligned}$$

(58a)

$$\begin{aligned} y&= \sum _{i=1}^L 2^{-i} u_i + \varDelta u\end{aligned}$$

(58b)

$$\begin{aligned} (x+\varDelta x, \beta _i, u_i)&\in {\mathcal {M}}(x+\varDelta x, \beta _i) \quad i \in \llbracket L \rrbracket \end{aligned}$$

(58c)

$$\begin{aligned} (\varDelta x, \varDelta u)&\in {\mathcal {M}}(\varDelta x)\end{aligned}$$

(58d)

$$\begin{aligned} \beta _i&\in \{0,1\} \quad i \in \llbracket L \rrbracket \end{aligned}$$

(58e)

$$\begin{aligned} \varDelta x&\in [0, 2^{-L}] \end{aligned}$$

(58f)

Since $\beta _i$ is binary, $u_i = \beta _i (x + \varDelta x)$ is represented exactly. Thus, the only possible error is introduced in the relaxation of $\varDelta y = \varDelta x^2$, which yields a maximum error of $2^{-2L-2}$, occurring when $\varDelta x = 2^{-L-1}$.

Now, the expected error of T-NMDT is the expected error from the relaxation of $\varDelta y = \varDelta x^2$. Modeling $\varDelta x$ as a uniform random variable within its bounds $[0,2^{-L}]$, and noting that the only overestimator from (56) is $y \le 2^{-L} \varDelta x$ we obtain expected overapproximation error

$$\begin{aligned} \begin{array}{rl} {\mathbb {E}}(2^{-L} \varDelta x - \varDelta x^2) &{}= \int _{0}^{2^{-L}} 2^L (2^{-L} \varDelta x - \varDelta x^2) \mathrm d \varDelta x\\ &{}= 2^L \int _{0}^{2^{-L}} (2^{-L} \varDelta x - \varDelta x^2) \mathrm d \varDelta x\\ &{}= 2^L (\frac{1}{6} (2^{-L})^3)\\ &{}= \frac{1}{6} 2^{-2L}. \end{array} \end{aligned}$$

(59)

Similarly, the expected underapproximation error can be computed as $\frac{1}{12} 2^{-2L}$.

Additional baseline computation summaries

In Table 9 we summarize the results of our baseline experiments stratified by the number of decision variables as in, e.g., Table 4 of Dey et al. [21].

Table 9 Computational results with instances stratified based on number of variables

General representations with sawtooth bases

The premise of our formulation is that the function $y=x^2$ can be arbitrarily closely approximated by a series of sawtooth functions. We discuss here if such approximations could conveniently apply to other polynomials.

In [54], the authors present a Fourier series-like method that leverages orthogonal triangular functions to derive a convergent class of $L_2$-optimal approximations for general functions on the interval $[-\pi ,\pi ]$. Define the periodic triangular functions

$$\begin{aligned} \begin{array}{rl} X(x) = {\left\{ \begin{array}{ll} \tfrac{\pi ^2 + 2\pi x}{8} &{} -\pi< x + 2\pi k \le 0, k \in {\mathbb {Z}}\\ \tfrac{\pi ^2 - 2\pi x}{8} &{} 0< x + 2\pi k \le \pi , k \in {\mathbb {Z}}\end{array}\right. }\\ Y(x) = {\left\{ \begin{array}{ll} \tfrac{\pi x}{4} &{} -\tfrac{\pi }{2}< x + 2\pi k \le \tfrac{\pi }{2}, k \in {\mathbb {Z}}\\ \tfrac{\pi ^2 - \pi x}{4} &{} \tfrac{\pi }{2} < x + 2\pi k \le \tfrac{3\pi }{2}, k \in {\mathbb {Z}}\end{array}\right. } \end{array} \end{aligned}$$

(60)

The authors then build their orthogonal basis functions using an orthogonal linear transformation of the basis

$$\begin{aligned} 1, X(x), Y(x), X(2x), Y(2x), \dots , X(nx), Y(nx). \end{aligned}$$

However, as with Fourier series approximations, this method has the limitation that all approximating functions are equal at the endpoints of the interval, resulting in a poor approximation for functions at which the endpoints are not equal. Thus, to obtain good approximations for $x^3$ on $[-\pi ,\pi ]$, we first add the linear function $-\pi ^2 x$ to enforce equality at the endpoints.

Then, applying this method to $x^2$ and $x^3 - \pi ^2 x$ on the interval $x \in [-\pi ,\pi ]$, we obtain the following numbers for the ($L_1$-error). Note that almost all of the Y(nx) functions are relevant for approximating $x^3 - \pi ^2 x$ (and no X(nx)’s), while only a few X(nx) functions (and no $Y(nx)'s$) are relevant for approximating $x^2$.

Table 10 Comparison of $L_1$-error

To investigate the outlook of sparsely approximating $x^3$ with triangular functions directly, we solved the following MIP to obtain the $L_1$-optimal triangular approximation to $x^3$ on the interval [0, 1] using re-scaled versions of the basis functions above, and explicitly including a linear shift. We discretely approximate the $L-1$ error via the error at uniformly-spaced points $x_1, \dots , x_{N_p} \in [0,1]$, allowing the inclusion of only $N_f$ triangular functions.

$$\begin{aligned} \begin{array}{rrll} \min &{}&{} \tfrac{1}{Np}\sum _{j = 1}^{N_p}t_j \\ s.t. &{} t_j &{}\ge \sum _{i = I}(\lambda _i f_i(x_j) + \lambda _0 x_j + f_c) - x_j^3 &{} \forall j\\ &{} t_j &{}\ge -(\sum _{i \in I}(\lambda _i f_i(x_j) + \lambda _0 x_j + f_c) - x_j^3) &{} \forall j\\ &{} -M \cdot \alpha _i &{}\le \lambda _i \le M \cdot \alpha _i &{} \forall i \ge 1\\ &{} \sum _{i=1}^N \alpha _i &{}\le N_f\\ &{} \lambda _i &{}\in [-M,M] &{} \forall i\\ &{} \alpha _i &{}\in \{0,1\} &{} \forall i \end{array} \end{aligned}$$

(61)

The result, shown in Fig. 4, suggests that it is not possible to use this triangular basis to obtain a similar-quality sparse approximation for $x^3$ as for $x^2$: the best achievable error rate for $x^3$ is roughly $O(N_f^{-2})$, compared to $O(2^{-2N_f})$ for the quadratic. See also Table 10 where we compare the convergence of the two approximations.

Fig. 4

The $L_1$-error of $x^3$ vs. the number of approximating triangular functions $N_f$. The equation of the regression line suggests an asymtotic error rate of roughly $O(N_f^{-2})$, compared to $O(2^{-2N_f})$ for the quadratic