Regularized decomposition of large scale block-structured robust optimization problems

Abstract

We consider a general robust block-structured optimization problem, coming from applications in network and energy optimization. We propose and study an iterative cutting-plane algorithm, generic for a large class of uncertainty sets, able to tackle large-scale instances by leveraging on their specific structure. This algorithm combines known techniques (cutting-planes, proximal stabilizations, efficient heuristics, warm-started bundle methods) in an original way for better practical efficiency. We provide a theoretical analysis of the algorithm and connections to existing literature. We present numerical illustrations on real-life problems of electricity generation under uncertainty. These clearly show the advantage of the proposed regularized algorithm over classic cutting plane approaches. We therefore advocate that regularized cutting plane methods deserve more attention in robust optimization.

Appendices

Appendix A: Brief overview of bundle methods in nonsmooth optimization

This appendix provides a brief description of bundle methods using standard notation and terminology, which are also the ones of this paper. For an in-depth description, we refer to the textbook (Hiriart-Urruty and Lemaréchal 1996b) and the recent article (de Oliveira et al. 2014).

We are interested in solving $\min _{x} f(x)$, where $f : \mathbb {R}^n \rightarrow \mathbb {R}$ is a nonsmooth convex mapping known through an “oracle” returning, for a given $x \in \mathbb {R}^n$ as an input, the value f(x) and a subgradient $g \in \partial f(x)$ (or approximations of them). Given a set of iterates $x^1,\ldots ,x^k$ for which the oracle has been called, we can set up the cutting plane model $\check{f}_k$ for f by defining

$$\begin{aligned} \check{f}_k(x):= \max _{\ell =1,\ldots ,k} \left\{ f(x^\ell ) + \left\langle g^\ell , x -x^\ell \right\rangle \right\} , \end{aligned}$$

where $g^\ell \in \partial f(x^\ell )$. The cutting-plane algorithm, which can be traced back to Kelley (1960), consists of defining the next iterate by solving $\min _{x} \check{f}_k(x)$, which can be formulated as a linear problem.

The cutting-plane method suffers from various drawbacks (including stability and memory issues) and an improved scheme is available: regularized cutting-plane methods or so-called bundle methods, dating back to the seminal work of Lemaréchal (1975). These methods use a quadratic stabilization to ensure that the next iterate is not far from the best current iterate, called stability center $\hat{x}$. The next iterate is the solution of

$$\begin{aligned} \min _{x} \check{f}_k(x) + \frac{1}{2t_k}\left\| x - \hat{x}^k\right\| ^2, \end{aligned}$$

(29)

which is a convex quadratic program (easy to solve by specialized methods Kiwiel 1986, 1994). The predicted decrease $\delta _k:= f(\hat{x}^k) - \check{f}_k(x^{k+1})$ is then used to update the stability center: if enough decrease is observed, i.e. the inequality

$$\begin{aligned} f(x^{k+1}) \le f(\hat{x}^k) - m\delta _k. \end{aligned}$$

(30)

holds, the new iterate $x^{k+1}$ becomes the new stability center $\hat{x}^{k+1}$; Otherwise the information from the oracle is just used to improve the cutting plane model. From a variational analysis perspective, bundle methods are thus damped proximal algorithms where the inner proximal iterations are stopped whenever sufficient decrease is observed on f.

Note that the proximal parameter $t_k$ is basically free: for exact oracles, convergence is guaranteed if $t_k$ stays within bounds (see Correa and Lemaréchal 1993). In practice, $t_k$ can be arbitrary if the stability center was changed; $t_k$ should not increase if the stability center was not changed; $t_k$ should strictly increase if the oracle is subject to imprecise calculations during noise detection steps [see e.g., van Ackooij and Sagastizábal 2014, eq. (2.16)]. A complete description of asymptotical rules can be found in van Ackooij and Frangioni (2016).

On top of stabilizing iterates, bundle methods also come up with the possibility of aggregating cutting-plane information into an aggregate cut. In theory, we can delete all but two cutting planes (the aggregate cut and the cut belonging to the current stability center).

Appendix B: Benders decomposition: terminology and references

The so-called Benders decomposition hinges on the observation that fixing some variables makes certain problems much easier. This observation, originally made for problems with an underlying linear structure (Benders 1962) potentially with stochastic aspects (Slyke and Wets 1969), was generalized to problems with some underlying convexity in the seminal work (Geoffrion 1972). We recall in this appendix the terminology and some important references about this classical method in operation research in a broad sense.

The slave problem consists in the resulting problem with the fixed variables. The master problem is related to finding the optimal allocation of the previously fixed variables. Since the domain of the value function need not be the whole space, the master problem is augmented with the so called feasibility cuts (i.e., an outer approximation of the convex domain of the value function). The master problem is also enriched with optimality cuts, as a matter of fact, a cutting-plane model of the value function. Benders algorithm is thus as a variant of the cutting-plane method, presented previously, and therefore subject to the well-known oscillation effect and slow convergence. Regularized Benders algorithms have been recently proposed (Zaourar and Malick 2014; van Ackooij et al. 2016a).

Many articles, including Sherali and Lunday (2013), Sahiridis et al. (2010), Wentges (1996), Van Dinter et al. (2013), Yang and Lee (2012) concretely deal with strategies for generating good (optimality) cuts, occasionally with a problem-dependent flavour. The authors of Codato and Fischetti (2006) are concerned with generating strong feasibility cuts and provide an important contribution in this view. Further improvements to the general scheme are concerned with a relaxation of the convexity assumption of the slave problem by allowing for integer variables (Caroe and Tind 1998) or the use of generalized “logic” duality (called inference duality) in Hooker and Ottosson (2003). In Zakeri et al. (2000), it is moreover suggested to inexactly solve the slave problem to compute some inexact but “cheap” cuts. In general, an appropriate choice of cuts (approximating the value function) is crucial, as illustrated in Kolokolov and Kosarev (2006).

Let us finally illustrate optimality and feasibility cuts more specifically on the following parametrized linear program:

$$\begin{aligned} \psi (x):= \min _y \left\{ q^\mathsf {T}y: Wy \le b - Tx \right\} . \end{aligned}$$

The inequality system $Wy \le b - Tx$ need not admit a solution for all x and that consequently $\psi (x) = \infty$ may happen for some x. Minimizing the convex mapping $\psi$ then amounts to handling also the hidden constraints $x \in \mathrm {dom}\,{\psi }$. By strong duality, we have $\psi (x) = \max _{\lambda \ge 0} \left\{ \lambda ^\mathsf {T}(b - Tx): W^\mathsf {T}\lambda \ge q\right\}$, so that any optimal dual multiplier $\lambda ^*$ satisfies $-T^\mathsf {T}\lambda ^* \in \partial \psi (x)$. When $Wy \le b - Tx$ does not admit a feasible solution, by Farkas Lemma an unbounded ray can be identified and used to define an outer linearization for $\mathrm {dom}\,{\psi }$. Note finally that, in the general non-linear situation, first order information about $\psi$ can also be identified with tools from nonsmooth analysis (e.g., Rockafellar and Wets 2009, Theorem 10.13), but feasibility cuts will need to be derived by solving an auxiliary problem typically involving the minimization of slack variables.

Appendix C: A rich but computationally cheap model of uncertainty

For our numerical experiments, we choose to use the uncertainty set $\mathcal {D}$ introduced by Minoux (2014). This so-called “state-space represented” model has a rich expressivity while being computationally easy to manipulate. This model has an exponential number of scenarios expressing temporal dependencies, while preserving a very fast computability by basic dynamic programming. We refer to Minoux (2014) more details and a complete analysis. This model is illustrated in Fig. 6. Yet, for the convenience of the reader, we describe here the main ideas briefly.

Fig. 6

Illustration of the uncertainty model used in the numerical experiments

At each time step $t \in \tau$, we set up a set of nodes $E_t$ containing both a value for $d_t$ and a weight $w_t$. This set of nodes is assumed to be sorted according to increasing values for $d_t$. Using this set of nodes we set up a graph (G, V), with $G = \bigcup _{t \in \tau } E_t$, such that V connects all nodes in $E_t$ to those in $E_{t+1}$ that are not further apart than H, i.e., node $i \in E_t$ is connected to node $j \in E_{t+1}$ if and only if their local labels $\left| \mathcal {L}(i)-\mathcal {L}(j) \right| \le H$. At least one of the nodes in $E_t$ is assumed to have a zero weight and H is assumed to be such that each node is connected to the node of zero weight. The set $\mathcal {D}$ is now given by all paths in (G, V) satisfying $\sum _{t \in \tau } w_t \le W_\mathtt{max}$ for a given maximum budget of uncertainty $W_\mathtt{max}$. As a consequence, $\left| \mathcal {D} \right|$ is huge, but computing the sup over $\mathcal {D}$ amounts to applying a simple 1-dimensional dynamic programming principle.