Ray projection for optimizing polytopes with prohibitively many constraints in set-covering column generation

Abstract

A recurrent task in mathematical programming consists of optimizing polytopes with prohibitively many constraints, e.g., the primal polytope in cutting-planes methods or the dual polytope in column generation (CG). We present a method to optimize such polytopes using the following intersection sub-problem: given ray ${\mathbf r}\in \mathbb {Z}^n$, find the maximum ${t^*}\ge 0$ such that ${t^*}{\mathbf r}$ is feasible. We interpret ${\mathbf 0_n}\rightarrow {\mathbf r}$ as a direction of growth from the origin ${\mathbf 0_n}$; ${t^*}{\mathbf r}$ is the intersection point between ${\mathbf 0_n}\rightarrow {\mathbf r}$ and the polytope boundary. To ease the exposition, the first part of the paper is devoted to the general context of a (primal or dual) polytope with rational data such that: (i) the intersection sub-problem can be effectively solved in practice for input rays ${\mathbf r}\in \mathbb {Z}^n$ and (ii) ${\mathbf 0_n}$ is feasible. By iterating over such rays ${\mathbf r}$, our method produces a sequence of feasible solutions ${t^*}{\mathbf r}$ that we prove to be finitely convergent. From Section 3 on, we focus on dual polytopes in CG formulations. Typically, if the CG (separation) sub-problem can be solved by dynamic programming (DP), so can be the intersection sub-problem. The proposed integer ray method (IRM) only uses integer rays, and so, the intersection sub-problem can be solved using a DP scheme based on states indexed by integer ray values. We show that under such conditions, the intersection sub-problem can be even easier than the CG sub-problem, especially when no other integer data is available to index states in DP, i.e., if the CG sub-problem input consists of fractional (or large-range) values. As such, the IRM can tackle scaled instances (with large-range weights) of capacitated problems that seem prohibitively hard for classical CG. This is confirmed by numerical experiments on various capacitated Set-Covering problems: Capacitated Arc-Routing, Cutting-Stock and other three versions of Elastic Cutting-Stock (i.e., a problem class that includes Variable Size Bin Packing).

Appendices

Appendix 1: Small LP optimized by iterating over rays

We here illustrate the convergence process the LP from Fig. 1, copied here to ease the reading. Importantly, the ray choices from this example do not accurately represent the real IRM choices: the ray generator is voluntarily simplified, as we only want to illustrate the general idea of iterating over rays rather then the details of our ray generator.

Step 1 IRM starts out with ${\mathbf r}_1={\mathbf b}=[1~1]$. The intersection sub-problem leads to:

• ${t^*}=5$;

• $c_a=15$ and ${\mathbf a}=[2~1]$, i.e., $2x_1+x_2\le 15$;

• ${{\mathbf x}_{\text {lb}}}=[5,5]$;

• ${{\mathbf x}_{\text {ub}}}=[0~15]$, i.e., the optimal solution over the polytope delimited by ${\mathbf x}\ge {\mathbf 0_n}$ and above “first-hit” constraint $2x_1+x_2\le 15$.

Step 2 As hinted above, the ray generator from this example is voluntary simplified⁵ and we consider that the next ray is ${\mathbf {r_\text {new}}}={{\mathbf x}_{\text {ub}}}=[0~15]$. After solving the intersection sub-problem, we obtain:

• ${t^*}=10/15$;

• $c_a=10$ and ${\mathbf a}=[0~1]$, i.e., $x_2\le 10$;

• ${{\mathbf x}_{\text {lb}}}=[0,10]$,

• ${{\mathbf x}_{\text {ub}}}=[2.5, 10]$, i.e., the above first-hit constraint separates the old ${{\mathbf x}_{\text {ub}}}=[0~15]$, thus updating ${{\mathbf x}_{\text {ub}}}=[2.5, 10]$. In the figure, we observe that ${\mathbf x}_{\text {ub1}}$ advances towards ${\mathbf x}_{\text {ub2}}$.

Step 3 IRM could take ${{\mathbf {r_\text {new}}}}=[3~10]$, using a rounding of the form $\lfloor {{\mathbf x}_{\text {ub}}}+\frac{1}{2}\mathbf 1_n \rfloor$. The intersection sub-problem leads to:

• ${t^*}=50/55=10/11$;

• $c_a=50$ and ${\mathbf a}=[5~4]$, i.e., $5x_1+4x_2\le 50$;

• ${{\mathbf x}_{\text {lb}}}=[\frac{30}{11},\frac{100}{11}]$;

• ${{\mathbf x}_{\text {ub}}}=[2~10]$.

Observe that ${\mathbf b}^\top {{\mathbf x}_{\text {lb}}}=\frac{130}{11}\approx 11.78$ and that ${\mathbf b}^\top {{\mathbf x}_{\text {ub}}}=12$, i.e., the gap is very small.

Step 4 IRM could consider ${\mathbf r}={{\mathbf x}_{\text {ub}}}=[2~10]$, solve the intersection sub-problem to find ${{\mathbf x}_{\text {lb}}}={{\mathbf x}_{\text {ub}}}=$OPT$(\mathcal {P})=[2~10]$.

Appendix 2: Algorithm for the CARP intersection sub-problem

The states from Sect. 4.2.3 are recorded as follows. For a given $v\in V$, a list data structure states $[v]$ keeps all states $(p_s, c_s, v)$ sorted in a lexicographic order, i.e., first ordered by profit and then by cost. In fact, for each integer profit $p>0$, one can access all states $(p,c_s,v)$ as if they were recorded in a separate sublist corresponding to a fixed profit $p$. Since all existing instances use integer edge lengths, we can consider that the deadheading traversal costs $c_s$ are also integer (they are sums of edge lengths). This is not a theoretical IRM requirement, but it is useful for reducing the number of states in practice.

The (rather self-explanatory) pseudocode is provided in Algorithm 4 and commented upon in the subsequent paragraphs. We emphasize the following key stages:

Stage 1: build continuous service segments Any required edge $\{u,v\}\in E_R$ can be used to augment (continue) a service segment ending in $u\in V$. Each repeat-until iteration of Stage 1 (Lines 3–14) considers all such segments (in Line 4) and tries to augment them to higher weights (increased load) by servicing any edge $\{u,v\}\in E_R$ (in Line 5). When all such service increases lead to higher weights exceeding $C$, the Stage 1 ends by breaking the repeat-until loop with the condition in Line 14. Also, at each new update of $s\in \mathtt{states} [v]$, we record the state $(p,c,u)\in \mathtt{states}[u]$ and the edge (i.e., $\{u,v\}$) that led to $s$. We thus build a network of precedence relations between states that is later useful to: (i) reconstruct an actual optimal route ${\mathbf a}$ in the end, (ii) eliminate 2-cycles (U-turns).⁶

Stage 2: continue with transfer segments For any $u,v\in V$ ($u\ne v$), a single transfer $u\rightarrow v$ may be required to link a service segment ending in $u$ to a service segment starting in $v$. This transfer can only increase the dead-heading cost, but not the weight (load). By chaining Stages 1 and 2, one can generate any sequence of service and dead-heading segments. The null service (no move) is added in Line 1 to allow Stage 2 to discover routes that actually directly start by dead-heading.

Stage 3: optimum update Check for better cost/profit ratios on complete routes (ending at $v_0$) in Lines 23–25. Stage 4: prune useless states Remove as many states as possible so as to break the main do-while loop (Lines 2–23) as soon as possible by triggering the condition in Line 33 (i.e., no state left). Given $v\in V$, a state $(p,c,v)\in \mathtt{states}[v]$ can be removed in one of the following cases:

• (A) $(p,c,v)$ was not updated by the last call of Stage 1 or Stage 2. This indicates that state $(p,c,v)$ has already generated other states in Stage 1 or 2 and any future use becomes redundant. However, the value $W(p,c,v)$ is recorded because it can still be useful: if state $(p,c,v)$ is re-discovered later, there is no use to re-insert it—unless the rediscovered weight is smaller than $W(p,c,v)$.

• (B) the cost $c$ is so high that the state $(p,c,v)$ can only lead to ratios cost/profit thare are guaranteed to stay above the current upper bound $t_{\text {ub}}$. For instance, if $t_\text {ub}=0$ and $c>0$, all states can be removed and the algorithm finishes very rapidly—at the second call of Line 33, after removing already-existing states using (A).

The only external routine is pMaxResidual($wRes$) in Stage 4, Line 28. It returns the maximum profit realized by a route ending at the depot using a capacity of no more then $wRes$ (for any starting point). We used the following upper bound for this profit: ignore the dead-heading costs (the vehicle can directly “jump” from one vertex to another when necessary) and obtain a classical knapsack problem with capacity $wRes$, weights ${\mathbf w}$ and profits ${\mathbf r}$. This upper bound can be calculated once (e.g., via DP) for all realizable values $wRes\in [0..C]$ in a pre-processing stage.

Appendix 3: Intermediate Lagrangean bounds in column generation

To (try to) make the paper self-contained, let us discuss in greater detail the Lagrangean CG lower bounds mentioned throughout the paper. The numerical Cutting-Stock experiments from Sect. 5.1.3 rely on the Farley bound, which is one of the specializations of the Lagrangean CG bound. While such CG bounds are a very general CG tool, their calculation can be different from problem to problem. To ease the exposition, we recall the main Set-Covering CG model from Sect. 3 and we present below both the primal (left) and the dual (right) programs:

$$\begin{aligned} \begin{array}{lll} &{}\min \sum _{}c_a\lambda _a&{}\\ {\mathbf x}: &{}\sum a_i\lambda _a\ge b_i,&{}\quad \forall i\in [1..n]\\ &{}\lambda _a\in \mathbb {R}_+ &{}\quad \forall {[c_a~{\mathbf a}]^\top \in \overline{\mathcal {A}}}\\ \end{array} \quad \begin{array}{lll} &{}\max {\mathbf b}^\top {\mathbf x}&{}\\ \mathbf {\lambda }:&{}{\mathbf a}^{\top }{\mathbf x}\le c_a,&{}\quad \forall [c_a,{\mathbf a}]\in \overline{\mathcal {A}},\\ &{}x_i\ge 0,&{}\quad i\in [1..n] \end{array} \end{aligned}$$

where all sums are carried out over all primal columns $[c_a,{\mathbf a}]\in \overline{\mathcal {A}}$.

We describe the Lagrangean bound on the primal (left) program, by reformulating ideas from work such as [6, §2.2], [25, §3.2], [19, §2.1] or [7, §1.2]. Each iteration of the CG process yields some dual values ${\mathbf x}$ that we use as Lagrangean multipliers to relax the primal set-covering constraints $\sum a_i\lambda _a\ge b_i$. More exactly, for any such fixed ${\mathbf x}$, the Lagrangean bound $\mathcal L_{{\mathbf x}}$ is simply obtained from the primal by introducing penalty $x_i$ for violating $\sum a_i\lambda _a\ge b_i$. Technically, the primal becomes: $\mathcal L_{{\mathbf x}}=\min _{\lambda } \left( \sum _{}c_a\lambda _a+{\mathbf x}^\top ({\mathbf b}-\sum {\mathbf a}\lambda _{\mathbf a})\right)$, where all sums are carried out over all primal columns $[c_a,{\mathbf a}]\in \overline{\mathcal {A}}$. After classical algebraic reformulations, we obtain the following relation between the CG optimum $\text {OPT}_{CG}$ and $\mathcal L_{{\mathbf x}}$.

$$\begin{aligned} \text {OPT}_{CG}\ge \mathcal L_{{\mathbf x}}= \min _{\lambda \ge \mathbf 0} \left( \sum (c_a - {\mathbf a}^\top {\mathbf x})\lambda _a\right) + {\mathbf b}^\top {\mathbf x}\ge \min _{\lambda \ge \mathbf 0} \left( m_{\text {rdc}} \sum \lambda _a \right) + {\mathbf b}^\top {\mathbf x},\nonumber \\ \end{aligned}$$

(6.1)

where $m_{\text {rdc}}=\min _{[c_a,{\mathbf a}]\in \overline{\mathcal {A}}}c_a - {\mathbf a}^\top {\mathbf x}\le 0$ is the minimum reduced cost at the current iteration. Assuming that a primal upper bound $u_b$ is known, one could add $\sum \lambda _a \le u_b$ to the primal; as such, $\sum \lambda _a\le u_b$ also holds in (6.1), which can that evolve to:

$$\begin{aligned} \text {OPT}_{CG}\ge {\mathbf b}^\top {\mathbf x}+ u_b\cdot m_{\text {rdc}}, \end{aligned}$$

(6.2)

We see that one requires an upper bound value $u_b$. This can be obtained, for instance, by classical primal heuristics. However, for pure Cutting-Stock, the fact that $c_a$ is always 1 leads to $\sum \lambda _a=\sum c_a\lambda _a\le \text {OPT}_{CG}$, i.e., one can use $u_b=\text {OPT}_{CG}$, which would transform (6.2) into $\text {OPT}_{CG}\ge {\mathbf b}^\top {\mathbf x}+ \text {OPT}_{CG}\cdot m_{\text {rdc}}$. The Farley lower bound $\mathcal L^F_{\mathbf x}$ follows immediately:

$$\begin{aligned} \mathcal L^F_{\mathbf x}=\frac{{\mathbf b}^\top {\mathbf x}}{1-m_{\text {rdc}}} \le \text {OPT}_{CG}. \end{aligned}$$

Since all our Elastic Cutting-Stock problems verify $c_a\ge 1,\forall [c_a,{\mathbf a}]\in \overline{\mathcal {A}}$, we could still use $u_b=\text {OPT}_{CG}$, simply because $\sum \lambda _a\le \sum c_a\lambda _a\le \text {OPT}_{CG}$; in this case, the above bound $\mathcal L^F_{\mathbf x}$ is still correct.

In any case, to exploit (6.2), one needs a primal upper bound $u_b$ on $\sum \lambda _a$. Generally speaking, to obtain high-quality upper bounds for other problems, one might need more specific modelling work.

Appendix 4: Cutting-stock instances: characteristics and references

See Table 5.

Table 5 Original Cutting-Stock instance files