On the safety of Gomory cut generators

Abstract

Gomory mixed-integer cuts are one of the key components in Branch-and-Cut solvers for mixed-integer linear programs. The textbook formula for generating these cuts is not used directly in open-source and commercial software that work in finite precision: Additional steps are performed to avoid the generation of invalid cuts due to the limited numerical precision of the computations. This paper studies the impact of some of these steps on the safety of Gomory mixed-integer cut generators. As the generation of invalid cuts is a relatively rare event, the experimental design for this study is particularly important. We propose an experimental setup that allows statistically significant comparisons of generators. We also propose a parameter optimization algorithm and use it to find a Gomory mixed-integer cut generator that is as safe as a benchmark cut generator from a commercial solver even though it generates many more cuts.

Appendices

Appendix A: Dive-and-Cut Solution-Generation phase

In this section, we describe the algorithm GenerateSolutions (see Algorithm 4) used to generate a set $S$ of ($\epsilon _{\text {abs}}, \epsilon _{\text {rel}}, 0$)-feasible solutions for a problem instance $P$. GenerateSolutions applies a Branch-and-Cut solver $\mathcal{B }$ to the instance at hand $P$ and acts whenever the solver discovers an integer solution. In Cplex, this can be achieved by a call to the function incumbentcallback().

Whenever a candidate solution $\tilde{x}$ is discovered, GenerateSolutions rounds each integer variable to the nearest integer, and checks the feasibility of this rounded solution. If it fails to satisfy the constraints within an absolute feasibility tolerance $\epsilon _{\text {abs}}$ and a relative feasibility tolerance $\epsilon _{\text {rel}}$, it is rejected.

If $\tilde{x}$ is not rejected, we use a rational LP solver to generate provably feasible solutions to the problem $P$ that have value $\tilde{x}_i$ for all integer variables $x_i$. First, we seek a provably feasible solution $x^*$ that minimizes the original objective function. If we are able to compute $x^*$, it is added to the set $S$ of feasible solutions. If its Euclidean distance to $\tilde{x}$ is less than $\epsilon _{\text {rel}}$, we are done and $\tilde{x}$ is accepted. Otherwise, we seek the point $x'$ in the feasible region of the LP that is closest to $\tilde{x}$ in $\ell _1$-norm. If $\Vert \tilde{x} - x'\Vert _2 \le \mathtt{{ZERO}}$, these two points are essentially the same and $x'$ is added to $S$. If $\Vert \tilde{x} - x'\Vert _2 \le \epsilon _{\text {rel}}$, $\tilde{x}$ is “feasible enough” according to our criteria and is added to $S$. In both cases, $\tilde{x}$ is accepted. Otherwise, it is rejected.

Note that GenerateSolutions uses the $\ell _1$-norm for the problem of finding $x'$ close to $\tilde{x}$ since this can be formulated as an LP using the usual reformulation: The objective function $\min _{x \in \mathbb{R }^n} \Vert \tilde{x} - x\Vert _1$ can be expressed in linear form by adding $n$ extra variables and $2n$ constraints: $\min _{w, x \in \mathbb{R }^n} \{ \mathbf{1 }_n w \; | \;\tilde{x} - x \le w, x - \tilde{x} \le w\}$. Since the $\ell _1$-norm overestimates the $\ell _2$-norm this guarantees that $x'$ is also at most $\epsilon _{\text {rel}}$ away from $\tilde{x}$ in $\ell _2$-norm. As a consequence, if such an $x'$ exists, the relative violation of any nonnegative linear combinations of rows of $Ax \ge b$ by $\tilde{x}$ is at most $\epsilon _{\text {rel}}$.

GenerateSolutions accomplishes several desirable goals. First, it does not modify the original instances. Second, given enough time, it finds an optimal solution (feasible according to our criteria) and adds it to the set $S$. Third, it potentially returns a large and diverse set of feasible solutions, with different values for the integer variables. This is valuable for our purposes, as this allows testing the validity of cutting planes with respect to solutions in different parts of the feasible region.

However, GenerateSolutions has also some drawbacks. First, the generated solutions are the nearest floating point representation (component-wise) of the feasible solutions computed in rational numbers. Therefore, they could be infeasible when the left-hand side of the constraints is computed in finite precision. Second, the generated solutions might be cut off in terms of absolute violation by linear combinations of the problem constraints. Using only a relative violation tolerance would address the second issue, as explained above. However, using also an absolute feasibility tolerance makes sense, because all available Branch-and-Cut codes have an absolute feasibility tolerance $\epsilon _{\text {abs}} > 0$.

1.1 A.1: Implementation

GenerateSolutions’s implementation is tied to Cplex 12.2 because it uses advanced Branch-and-Cut functions. In particular, we use the incumbentcallback() function to intercept the discovery of feasible solutions and run our implementation of Algorithm 4. The feasibility and integrality tolerances for Cplex were set to $10^{-9}$, the smallest feasibility tolerance allowed by Cplex.² The rational LP solver used by GenerateSolutions is QSopt_ex [4].

We parameterize the Branch-and-Cut algorithm of Cplex as follows. The integrality gap allowed for optimality is set to 0 (both relative and absolute). The time limit is set to 6 h, the number of parallel threads to 2, and SolutionPoolIntensity is set to 2. We disable rescaling of the LP matrix, in order to avoid discovery of solutions that are infeasible for the unscaled problem (Cplex error code: OptimalInfeas). If the instance is solved to optimality before the 6 hours time limit, we call the CPXpopulate() procedure to generate more solutions until the time limit is hit or an additional 100 solutions are found. The strategy for replacing solutions in the pool when it is full is set to Diversify.³

The experiments in this section are run on a machine equipped with an AMD Opteron 4176 HE processor clocked at 2.4GHz and 48GB RAM, running Linux.

1.2 A.2: Computational experiments on MIPLIB instances

Our initial test set contains all instances from MIPLIB3 [7], MIPLIB2003 [2], and the Benchmark set of MIPLIB2010 [21] beta (downloaded March 2011) for a total of 169 instances. Ten instances are eliminated, as GenerateSolutions does not find any feasible solution for them. We are left with the 159 instances listed in Table 13. On the satellites1-25 instance, more than $10^6$ solutions are found during the initial Branch-and-Cut; we keep the best solution found and an additional 1,499 solutions selected at random.

Table 13 Number of feasible solutions found by GenerateSolutions on each instance

The vast majority of the solutions found by GenerateSolutions are feasible with very small tolerances. In Fig. 14 we report a histogram of the maximum absolute violation for the generated solutions, computed as $\max _{i \in [m]} \{b_i - a^i x^*\}$. The maximum violation for 62.99 % of the solution is 0, i.e., they are feasible even when checked using finite precision and compensated summation [20]. We note that the use of compensated summation here has a small effect, increasing the fraction of solutions with a 0 violation by 0.27 %.

Fig. 14

Maximum absolute violation of the solutions found by GenerateSolutions

For 98.8 % of the solutions the maximum violation is smaller than $10^{-11}$. We eliminate the solutions for which the maximum violation is $10^{-9}$ or more.

For 152 instances out of 159 the optimal solution is known, therefore we can analyze the objective value of the solutions found by GenerateSolutions on these instances. We compute the relative distance from the known optimum $f^*$ of the best solution value $\bar{f}$ among the solutions accepted by GenerateSolutions, using the formula: $(\bar{f}-f^*)/|f^*|$ if $f^*\ne 0$, or $(\bar{f}-f^*)$ otherwise. In 138 cases out of 152, this value is $<0.5\,\%$, hence we found a solution that satisfies our criteria and can be considered optimal. For the remaining 14 cases, the average relative distance is 231.02 %. This large value is due to a small number of instances where GenerateSolutions returns solutions of poor quality: markshare1 (200 % away from the optimum), markshare2 (2,100 % away from the optimum), satellites1-25 (700 % away from the optimum).

The 51 instances in bold face in Table 13 form the Failure Set. The selection is discussed in Sect. 4.2. The 69 instances in italics in Table 13 form the Extended Set used to validate our results on the Failure Set. The Extended Set contains all instances of Table 13 that are not part of Failure Set and for which 300 dives require $<24$ h of CPU time and generate $<30$ failures of Type 3.

The instances, solutions and code used in this paper are available on the website of the authors.

Appendix B: Parameter sensitivity

This section contains a sensitivity analysis of the failure and rejection rates when small changes are applied to the parameters of the cut generator Bestgen from Table 7, denoted by $g^*$ in this section. Experiments were run in the Condor setup.

We start with the parameterization $p^*$ of $g^*$ given in Table 7 and we investigate changes in a single parameter at a time. For each parameter $i$, we want to find a range $R_i$ such that changing the value in $p^*$ of $p^*_i$ to any value in $R_i$ yields a generator such that both a Friedman test on the rejection rate and a Friedman test on the failure rate deem the generator comparable to $g^*$. We then determine the variation of the failure rate and cut rejection rate for values slightly outside of $R_i$.

The algorithm we use is a simple sampling algorithm. It maintains an interval $\underline{R_i}$ containing the convex hull of all tested values of $p_i$ for which the modified generator is deemed similar to $g^*$. The initial interval $\underline{R}_i$ is the single point $p^*$. The algorithm also maintains the smallest interval $\overline{R}_i$ containing $\underline{R}_i$ such that one tested point on each side of $\underline{R}_i$ is deemed different between $g^*$ and the modified generator. The initial interval $\overline{R}_i$ is chosen as the initial range of parameter $i$ used in Sect. 6.4. However, if we already know from previous experiments one value $p_i$ smaller (resp. larger) than $p^*_i$ such that the corresponding cut generator is not equivalent to $g^*$, we remove all values smaller (resp. larger) than $p_i$ from the initial interval $\overline{R}_i$.

At each iteration, we select $k$ parameter values $p_1,\ldots ,p_k \in \overline{R}_i{\setminus }\underline{R}_i$, compare the corresponding cut generators with $g^*$ and update both $\underline{R}_i$ and $\overline{R}_i$. The algorithm continues until we are satisfied with the gap between $\underline{R}_i$ and $\overline{R}_i$. We use $k = 10$, except when the gap between $\underline{R}_i$ and $\overline{R}_i$ is very small. In that case, we use a smaller value for $k$. The tested values $p_1,\ldots ,p_k$ are equally spaced in $\overline{R}_i{\setminus }\underline{R}_i$, with $k/2$ points on each side of $\underline{R}_i$. We use a logarithmic scale for all parameters that have a logarithmic scale in Table 3.

A similar algorithm is used to compute intervals $\underline{F_i}$ and $\overline{F_i}$ such that all generators in $\underline{F_i}$ have a safety level similar to $g^*$ according a Friedman test on the failure rate, without regard to the rejection rate. The interval $\overline{F}_i$ contains $\underline{F}_i$ and one tested point on each side of $\underline{F}_i$ for which the modified generator is deemed different to $g^*$ in term of safety.

Results are reported in Table 14. The columns are as follows. “$p^*$” contains the optimal value of the parameter in cut generator #1 of Table 7. For the parameters that are not reported in Table 7, we use the value corresponding to disabling the cut modification or numerical check that employs the parameter. For the _LUB parameters, we set the value to the corresponding non-LUB parameter. In this table, LUB was set to $10^3$ (see Sect. 5.1). Columns “$\underline{R}$”, “$\overline{R}$”, “$\underline{F}$”, and “$\overline{F}$” contain the four intervals of interest for each parameter. “Increase” indicates the direction along which the cut rejection rate increases; the column contains “–” if our experiments did not provide enough information for computing the direction.

Table 14 Sensitivity analysis of the parameters of the reference cut generator