Preface

This special issue of Mathematical Programming series B collects papers authored (or co-authored) by researchers who attended the second Oberwolfach workshop on MINLP, titled “Mixed-integer Nonlinear Optimization: a hatchery for modern mathematics”, and co-organized by the guest editors of this special issue. The workshop took place in early june 2019. A summary of the proceedings of the workshop can be found in [40]. The workshop was organized around three main sub-topics of MINLP: hierarchies of approximations, mixed-integer optimal control (MIOC), and uncertainties. Other contributions outside of these topics were collected into a generic “other areas” category.

At the risk of boring the MINLP specialist, but for the benefit of the occasional reader who might stumble upon this preface, we say that MINLP is the acronym of “Mixed-Integer Nonlinear Programming”, which describes a large subclass of Mathematical Programming¹ (MP) problems: notably, those containing both continuous and integer decision variables, and nonlinear objective function and/or constraints.

1 An incredibly short summary of MINLP history

In all human sciences, when a new result becomes important in a community, it is likely that the scientific environment was “ripe”: and therefore it is difficult to mark a single publication as “the first”. Accordingly, it is hard to tell when MINLP was officially born. If we count bounded Integer Polynomial Programming (bIPP) as a signficant representative class for MINLP, the first reference we found is the paper [28] by P.L. Ivanescu (who later took the name of Peter Hammer), which cites [8] as the main inspiration (a few years later, P. Hansen introduced a tree-like search for bIPP in [26]). If we want more generality than polynomials, but accept separability between linear and nonlinear parts of the problem, we can consider [9], by J. Benders: the proposed solution algorithm exploits the obvious decomposition in linear and nonlinear parts.

As a side note, R. Jeroslow [29] showed the undecidability of MINLP by exploiting an Integer Quadratically Constrained Program (IQCP) and leveraging the Davis-Putnam-Robinson-Matiyasevich theorem [36].

E. Balas, in a sequence of several papers during the late 1960s and early 1970s, introduced a duality theory for Mixed-Integer Programming (MIP), based on previous work by J. Stoer. The last paper in this sequence [6] bears the title “A duality theorem and an algorithm for (Mixed-Integer) Nonlinear Programming”, which we think might be the first reference to the term MINLP. Balas’ duality theory for MIP is based on a minimax principle. The suggested algorithmic approach is based on Benders’ decomposition. We refer to the bibliographies in Balas’ papers cited above for more references.

Currently, for generic MINLP for which no structure is known in advance, the best exact² algorithm for solving MINLP is the spatial Branch-and-Bound (sBB) algorithm. The sBB algorithm is a variant of the Branch-and-Bound (BB) algorithm, which was introduced in [32] and extended to (separable) Nonlinear Programming (NLP) in [17].

1.1 The chemical engineering boost

The motivation to construct exact algorithms for MINLP came from Chemical Engineering, and in particular from Process Synthesis, where local solutions may lead to unsafe chemical plants. I. Grossmann was probably the main actor in the field of exact algorithms for MINLP coming from Chemical Engineering: he admits that during his Ph.D. studies in the late 1970s, at the Centre for Process Systems Engineering, Imperial College, he was influenced by an academic environment that was clearly ready to consider and model problems using nonlinear functions of both continuous and binary variables.

Grossmann’s Ph.D. supervisor, R. Sargent, apparently mentioned MINLP as the type of accurate model for which no solution technique yet existed at the time. Some techniques in the literature for nonconvex NLPs had in fact been conceived to address problems with a limited number of binary variables. Specifically, in [24], the authors sketch what looks like a BB for convex MINLP (cMINLP), as well as an extension of the Generalized Benders Decomposition (GBD) [23] to MINLP. The first occurrence of the Outer Approximation (OA) algorithm for cMINLP can be ascribed to a presentation by Grossmann at the 1983 TIMS/ORSA meeting in Chicago. By the late 1980s, Grossmann was publishing on MINLP with his own Ph.D. students [16, 31].

The 1990s saw the development of MINLP theory and solution methods starting from the mathematics of local optima of NLPs. Remarkable advances in this sense were made by: (1) R. Fletcher and his Ph.D. student S. Leyffer, who defined algorithms and implemented solvers based on both OA and BB for convex MINLP [18, 19]; (2) C. Floudas and his Ph.D. students, who introduced the generic $\alpha$BB technique for lower bounding nonconvex functions [1, 2, 4, 35]; (3) C. Pantelides and his Ph.D. students [7, 30, 50,51,52] at Imperial College, who introduced some symbolic computation techniques in order to algorithmically construct a convex relaxation of any factorable MINLP [37]; (4) N. Sahinidis and his Ph.D. students [45, 46, 49], who produced BARON, the first commercial MINLP solver that is still being developed [47]; (5) T. Westerlund and his collaborators [54], who developed a cutting-plane algorithm for convex MINLP. Grossmann and his collaborators continued to work on MINLP theory and methods [3, 44, 55]. This list is by no means exhaustive. More information can be found in authored and edited books from that period [10, 14, 20, 21, 25, 27].

1.2 Mixed-integer optimal control

An interesting case arises in the context of dynamic systems, where the optimization model contains differential equations. On the one hand such mixed-integer optimal control (MIOC) problems are a generalization of MINLPs, because the problem formulations allow for trivial special cases with $n_x=0$ differential states that are formally equivalent to a MINLP. On the other hand, a discretization of controls and states is one possible solution approach and results in a specifically structured and usually high-dimensional MINLP. This ambiguity in specifying the relation between MINLP and MIOC is reflected in the challenges (fine discretizations contribute to the curse of dimensionality) and opportunities (using smoothing properties in function spaces) that are used in several contributions in this special issue. It is important if the discretization is done before or after derivation of necessary conditions of optimality.

One related interesting property in optimal control is the possibility of an infinite number of switches in optimal solutions. This behavior is referred to as chattering in the optimal control community, [56]. The first example of an optimal control problem exhibiting chattering behavior was given by [22]. In the engineering community chattering behavior is also called Zeno’s phenomenon. This refers to the probably first appearance of hybrid systems in the literature and the great ancient philosopher Zeno of Elea. Zeno of Elea was a pre–Socratic Greek philosopher of southern Italy and a pupil of Parmenides. He is mostly known for his 40 paradoxes, among which the most famous are The Dichotomy (Motion is impossible since “that which is in locomotion must arrive at the half-way stage before it arrives at the goal.”), The Arrow (“If everything when it occupies an equal space is at rest, and if that which is in locomotion is always occupying such a space at any moment, the flying arrow is therefore motionless.”), and The Achilles (“In a race, the quickest runner can never overtake the slowest, since the pursuer must first reach the point whence the pursued started, so that the slower must always hold a lead.”). These paradoxes can be found, e.g., in Physics of [5], VI:9, 239. Zeno of Elea was the first to draw attention to the apparent interpretational problems occurring whenever an infinite number of events has to take place in a finite time interval.

Control problems for switched systems have been treated in different communities and branded differently. In this special issue the name mixed-integer optimal control (MIOC) prevails. In addition to MIOC and optimal control of switched systems, alternate names for the same or similar problem classes have been established, such as mixed-logic dynamic optimization, mixed-integer programming for control, bang-bang control, optimal control of hybrid systems, or mixed-integer PDE constrained optimization. Moreover, there are many different aspects which are usually considered for a more detailed classification of MIOC problems, e.g., the underlying dynamics (ordinary differential equations, differential-algebraic equations, partial differential equations), the type of switches (explicit or implicit), the problem structure (e.g., linear, linear-quadratic, or nonlinear), further restrictions (state, mixed control-state, combinatorial, vanishing, or no constraints), open-loop (feedforward) versus closed-loop (feedback) control, uncertainties and data-driven learning, and last but not least the algorithmic approach.

There are three generic approaches to solve model-based optimal control problems. We quickly survey them and comment on extensions for the MIOC case, i.e., the additional integrality requirement on some of the control functions. First, Dynamic Programming going back to the fundamental work of Bellman [8] seems to be particularly suited for a treatment of integer variables, because of the enumerative approach (it even inspired general MINLP research, as mentioned above). Optimal control problems on short time horizons (e.g., with constant control function value) can be solved by simple enumeration of the discrete choices, giving even an advantage compared to an optimization over continuous feasible sets. However, the approach suffers in general from the so-called curse of dimensionality, an exponential increase in runtime when the state dimension increases.

Second, indirect methods, also known as the first-optimize then-discretize approach, use necessary conditions of optimality in function space and solve the resulting boundary value problem numerically in a second step. The maximum principle in its basic form, also sometimes referred to as minimum principle, goes back to the early fifties and the works of Hestenes, Boltyanskii, Gamkrelidze, and of course Pontryagin [43]. Precursors of the maximum principle as well as of the Bellman equation can already be found in Carathéodory’s book of 1935, compare [42] for details. The maximum principle states the existence of adjoints ${\lambda }^*: [t_0,t_{\text {f}}] \mapsto \mathbb {R}^{n_x}$ that satisfy adjoint differential equations and transversality conditions. The optimal control ${u}^*: [t_0,t_{\text {f}}] \mapsto \mathbb {R}^{n_u}$ is characterized as the pointwise maximizer of the Hamiltonian function. This concept can also be transferred to integer controls using theorems which do not assume a locally connected feasible set for the controls, usually referred to as global maximum principles. A global maximum principle was used for disjoint control sets and solved numerically via the method of Competing Hamiltonians in the work of Bock and Longman in the early 1980s, [12]. To our knowledge this was the first time that a global maximum principle was applied to solve a practically relevant MIOC problem.

The third generic approach, direct or first-discretize then-optimize methods have become very popular for most practical control problems. The general idea here is to discretize the control functions first, e.g., with a piecewise constant approximation on a given grid. In a second step, the Karush-Kuhn-Tucker theorem and finite-dimensional optimization algorithms are used for the search of candidate solutions. There are multiple variants of how to solve the differential equations by means of shooting methods or collocation. Historically, shooting and collocation methods were first developed for boundary value problems [15, 38] and then adapted by Sargent [48], Bock [13], Biegler [11], and others to the direct approach. Note that R. Bulirsch was an unofficial advisor of G. Bock’s PhD thesis, J. Stoer edited the book in which Sargent’s publication appeared, Bulirsch and Stoer wrote one of the most important textbooks in numerical analysis together [53], and many MINLP protagonists at Carnegie Mellon were specifically interested in dynamic (Chemical Engineering) applications—indicating the close interaction of the MINLP and MIOC communities from the very beginning. The main challenge for direct approaches for MIOC is the high dimensionality of the resulting MINLP. As we see in several contributions to this issue, there are many ways to use arguments and concepts in function space that justify error-controled decompositions and efficient numerical schemes, though.

The years after 2000 saw the confluence of several of the trends we discussed, namely early mathematical programming and structured applications (mainly from chemical engineering). Given the broad interest, MINLP was labelled a “hot topic” by several academic communities, and many advances were made. We are skipping details because the 2000s are too close to the present time to be branded “history”, and because a preface must be limited in space. See the edited books [33, 34, 39, 41] and of course the literature surveys of the following publications for more information.

2 The contents of this issue

This issue contains ten articles, reviewed by at least two referees. As we wrote at the outset, the workshop was organized around three main MINLP topics (hierarchies of approximations, MIOC, uncertainties), and “other areas”. In fact, half of the papers we accepted are in MIOC, one is about hierarchies of approximations, and the rest are about other MINLP areas. Accordingly, we organized the contents in two broad categories: MINLP, and MIOC.

The first five papers are in the MINLP, and the last five papers are in the MIOCP category.

• 1. “Quadratic optimization with switching variables: the convex hull for $n=2$”, by S. Burer and K. Anstreicher. This article is representative of a very active area of MINLP research, which provides an extension to the analysis of facets for often-used and problematic sets in MILP. Many reasonably simple mixed-integer nonlinear sets have been analysed in MINLP so far — mostly involving multilinear and/or quadratic forms in mixed-integer variables. The present paper deals with the problem of finding the convex hull of the set $\{(x,x{x}^{\top },y{y}^{\top })\;|\;0\le x\le y\in \{0,1\}^n$ for $n=2$. Such sets are crucial to model occurrences where the binary y variables activate or de-activate the continuous variables x (e.g. production at level $x_i>0$ only occurs whenever the i-th facility is active, i.e. $y_i=1$, otherwise $x_i=0$ since $y_i=0$).

• 2. “Near-optimal analysis of Lasserre’s univariate measure-based bounds for multivariate polynomial optimization”, by L. Slot and M. Laurent. This article concerns hierarchies of approximation. These hierarchies are composed by sequences $R_1,R_2,\ldots$ of ever tighter convex relaxations of Polynomial Programming (PP) problems in minimization form. Each $R_r$ provides a lower bound to the original PP formulation P, so that $\mathsf {val}(R_r)\le \mathsf {val}(R_{r+1})\le \ldots \le \mathsf {val}(P)$. Such hierarchies are useful to compute bounds to the optimal objective function value of P, which are in turn useful within sBB algorithms. The issue discussed in this paper concerned the speed of convergence of the sequence to $\mathsf {val}(P)$. Specifically, $R_r$ is a relaxation derived by sum-of-squares of polynomials of degree r. The main result in this paper is that the sequence converges at a rate $O(\log ^2 r/r^2)$.

• 3. “Outer approximation for Global Optimization of Mixed-Integer Quadratic Bilevel Problems”, by T. Kleinert, V. Grimm, and M. Schmidt. Bilevel programming includes MP formulations P where some of the constraints are of the form “the decision variable vector x of P must be an optimum of the auxiliary MP formulation Q”. Thus, P is the upper level problem, and Q is the lower level problem. Bilevel programming is useful in order to model leader/follower interactions, as well as games. Bilevel programs are notoriously difficult, both from the theoretical and practical points of view. This paper discusses the case where P is a convex Mixed-Integer Quadratic Program (cMIQP) and Q is a convex Quadratic Program (cQP) with continuous variables. Several OA-based solution methods are proposed.

• 4. “The confined primal integral”, by T. Berthold and Z. Csizmadia. This is a computational paper. It proposes a new performance measure to compare MP solvers and heuristics in trade-off behaviour (e.g. faster termination vs. better solution quality).

• 5. “Mixing convex-optimization bounds for maximum-entropy sampling”, by Z. Chen, M. Fampa, A. Lambert, J. Lee. The maximum-entropy sampling problem is a fundamental and challenging combinatorial optimization problem, with application in spatial statistics. It asks to find a maximum-determinant order-s principal submatrix of an order-n covariance matrix. Many of the known upper bounds for the optimal value are based on convex optimization. The authors present a methodology for “mixing” these bounds to achieve better bounds.

• 6. “Compactness and Convergence Rates in the Combinatorial Integral Approximation Decomposition”, by C. Kirches, P. Manns, and S. Ulbrich. The authors formulate a general approximation result for optimization problems, which feature discrete and distributed optimization variables, and which are governed by a compact control-to-state operator. By applying the result to an application from signal processing they show that the developed theory can also be applied beyond optimal control problems.

• 7. “Penalty alternating direction methods for mixed-integer optimal control with combinatorial constraints”, by S. Goettlich, F. Hante, A. Potschka, and L. Schewe. The authors propose a penalty and alternating direction of multipliers method for MIOCs motivated by an exactness result. They illustrate its performance for the optimization of electric transmission lines with switching of the network topology.

• 8. “Mixed-Integer Optimal Control Problems with switching costs: A shortest path approach”, by F. Bestehorn, P. Manns, C. Kirches, and C. Hansknecht. In this paper switching costs in MIOCs are considered. Using an innovative reformulation into a shortest path problem on a parameterized family of directed acyclic graphs, the authors provide computational bounds. The efficacy of the approach is demonstrated by a comparison with an integer programming approach. comparison with an integer programming approach on a benchmark problem.

• 9. “Mixed-integer optimal control under dwell time constraints”, by C. Zeile, N. Robuschi, and S. Sager. The author consider the case were activated controls need to stay active for a certain while, which is often the case in practical applications. Heuristic solutions provide upper bounds on the objective function value of a mixed-integer linear program, which provide a priori error estimates for the MIOC problem. For the novel rounding algorithms also numerical results are presented.

• 10. “A Solution Framework for Linear PDE-Constrained Mixed-Integer Problems”, by F. Gnegel, A. Fuegenschuh, M. Hagel, S. Leyffer, and M. Stiemer. In this paper different approaches to MIOC with underlying linear partial differential equations are considered. One approach uses a preprocessing step which eliminates the states from the optimization problem. In the second approach certain constraints are just imposed on demand via constraint callbacks. Numerical experiments illustrate the results.

3 Some editorial information

Fifteen papers were received for this issue, all with at least one coauthor who participated in the 2019 Oberwolfach workshop on MINLP. We had to deal with three conflicts of interest in accepted papers, all concerning editors as authors. S. Sager, guest editor of this issue, is co-author of an accepted paper: his paper was handled by S. Leyffer, the Editor-in-Chief of MPB. In turn, S. Leyffer was the co-author of an accepted paper: S. Sager handled this paper outside of the MPB editorial system. J. Lee, Editor-in-Chief of MPA, was also the co-author of an accepted paper: S. Sager handled this paper outside of the MPB editorial system.