A novel parallel combinatorial algorithm for multiparametric programming

Abstract

Multiparametric programming and control has received a lot of attention in the past twenty years with significant advances reported in the open literature. Most existing algorithms for multiparametric programming typically suffer from a computational slowdown with respect to the number of parameters, constraints and variables resulting from the large expansion of active set combinations that must be considered. In this work, we present a novel parallel combinatorial algorithm for the solution of multiparametric Quadratic programs (mpQP) and multiparametric Linear Programs (mpLP) (i), generating better exploration rules that greatly reduces the number of active set combinations that must be considered and (ii) a high-performance parallelization scheme for this algorithm that scales well with the number of cores. The proposed algorithm is validated on numerous computational examples, including scaling analysis of the parallel algorithm up to 48 CPU cores and generating optimal Pareto fronts for the Markowitz Portfolio Selection Problem. In large problem instances, speedups of over a factor of 500 are observed.

Appendices

Appendix A: Bounds on slack variables

The upper bound of a slack variable, $s_i$, can be calculated via the LP in Eq. A1. This bound can directly be used in as the value of the big-M constraint relating to this variable $M^{s_i}$. The lower bound can be calculated by minimizing instead.

$$\begin{aligned} \max _{x,\theta ,s}\quad \quad s_i \end{aligned}$$

(A1a)

$$\begin{aligned} \text {s.t. }\quad A_Ex&= b_E + F_E\theta \end{aligned}$$

(A1b)

$$\begin{aligned} Ax +s&= b_i + F_i\theta \end{aligned}$$

(A1c)

$$\begin{aligned} s&\ge 0 \end{aligned}$$

(A1d)

$$\begin{aligned} A_\theta \theta&\le b_\theta \end{aligned}$$

(A1e)

$$\begin{aligned} x\in \mathbb {R}^m,&\theta \in \mathbb {R}^k, s\in \mathbb {R}^t \end{aligned}$$

(A1f)

Appendix B: Feasibility of an active set

To check the feasibility of an active set combination, $\mathcal {I}$, the LP described in Eq. B1 is solved. If this LP is feasible then the active set combination, $\mathcal {I}$, is feasible otherwise this is infeasible. The set $\mathcal {J}$ is the set of inactive constraints, denoting the compliment of the active set, $\mathcal {I}^C$. This is in essence checking if the polytope generated by the active set is nonempty.

$$\begin{aligned} \min _{x,\theta }\quad \quad 0 \end{aligned}$$

(B1a)

$$\begin{aligned} \text {s.t. }\quad A_Ex&= b_E + F_E\theta \end{aligned}$$

(B1b)

$$\begin{aligned} A_ix&= b_i + F_i\theta , \quad \forall i \in \mathcal {I} \end{aligned}$$

(B1c)

$$\begin{aligned} A_jx&\le b_j + F_j\theta , \quad \forall j \in \mathcal {J} \end{aligned}$$

(B1d)

$$\begin{aligned} A_\theta \theta&\le b_\theta \end{aligned}$$

(B1e)

$$\begin{aligned} x\in \mathbb {R}^m,&\theta \in \mathbb {R}^k, \end{aligned}$$

(B1f)

Appendix C: Optimality of an active set

To check the optimality of an active set, $\mathcal {I}$, the set $\mathcal {J} = \mathcal {I}^C$ as in appendix section B. Then, the KKT conditions of the multiparametric program are manipulated. Firstly, given a known active set set then the slack variables associated with every constraint in the active set are by definition zero, and in addition all multipliers associated with inactive constraints must also be zero as depicted in Eq. C1. This insight removes to need to explicitly include the complimentary conditions of the pKKT conditions when evaluating the optimality any specific active set combination.

$$\begin{aligned} i \in \mathcal {I} \implies s_i = 0 \end{aligned}$$

(C1a)

$$\begin{aligned} i \notin \mathcal {I} \implies \lambda _i = 0 \end{aligned}$$

(C1b)

Variables with hats, simply refer to the rows relating the values of the active set.

$$\begin{aligned} \min _{x,\theta , \lambda ,\mu , s} \quad \quad&0 \end{aligned}$$

(C2a)

$$\begin{aligned} \text {s.t. }\quad Qx + H\theta +&c + A_E^T\mu + \hat{A}^T\lambda = 0 \end{aligned}$$

(C2b)

$$\begin{aligned} A_Ex&= b_E+F_E\theta \end{aligned}$$

(C2c)

$$\begin{aligned} \hat{A}x&= \hat{b}+\hat{F}\theta \end{aligned}$$

(C2d)

$$\begin{aligned} A_jx +s_j&= b_j + F_j\theta , \quad \forall j \in \mathcal {J} \end{aligned}$$

(C2e)

$$\begin{aligned} s_j&\ge 0, \quad \forall j \in \mathcal {J} \end{aligned}$$

(C2f)

$$\begin{aligned} \lambda _i&\ge 0, \quad \forall i \in \mathcal {I} \end{aligned}$$

(C2g)

$$\begin{aligned} A_\theta \theta&\le b_\theta \end{aligned}$$

(C2h)

$$\begin{aligned} x\in \mathbb {R}^m, \theta \in \mathbb {R}^k,&s\in \mathbb {R}^{|\mathcal {J}|}, \lambda \in \mathbb {R}^{|\mathcal {I}|}, \mu \in \mathbb {R}^t \end{aligned}$$

(C2i)

Appendix D: Constructing a critical region from an active set

Once an active set combination is checked to be both feasible and optimal, then the critical region can be constructed. Given the statonary condition and the active set, this system can be inverted and the parametric solutions can be found for mpQPs. The matrix inverse in Eq. D1b can be effectively computed by taking the block matrix inverse.

$$\begin{aligned} \begin{bmatrix} Q &{} A^T_E &{} \hat{A}^T\\ A_E &{} 0&{} 0\\ \hat{A} &{}0 &{}0\\ \end{bmatrix}\begin{bmatrix} x \\ \mu \\ \lambda \end{bmatrix}&= -\begin{bmatrix} H\\ F_E\\ \hat{F} \end{bmatrix}\theta -\begin{bmatrix} c\\ b_E\\ \hat{b} \end{bmatrix} \end{aligned}$$

(D1a)

$$\begin{aligned} \begin{bmatrix} x(\theta ) \\ \mu (\theta ) \\ \lambda (\theta ) \end{bmatrix}&= -\begin{bmatrix} Q &{} A^T_E &{} \hat{A}^T\\ A_E &{} 0&{} 0\\ \hat{A} &{}0 &{}0\\ \end{bmatrix}^{-1}\left( \begin{bmatrix} H\\ F_E\\ \hat{F} \end{bmatrix}\theta +\begin{bmatrix} c\\ b_E\\ \hat{b} \end{bmatrix}\right) \end{aligned}$$

(D1b)

In the case of mpLPs a more direct solution strategy can by used. As the number of active constraints is equal to the number of deterministic variables, and that the constraints are linearly independent, implies that a direct inversion of the gradients of the active set can be done to calculate $x(\theta )$ and $\lambda (\theta )$.

$$\begin{aligned} \begin{bmatrix}A_E\\ \hat{A}\end{bmatrix}x&= \begin{bmatrix}b_E \\ \hat{b}\end{bmatrix} + \begin{bmatrix}F_E\\ \hat{F}\end{bmatrix}\theta \end{aligned}$$

(D2)

$$\begin{aligned} x(\theta )&= \begin{bmatrix} A_E\\ \hat{A}\end{bmatrix}^{-1}\left( \begin{bmatrix} b_E \\ \hat{b}\end{bmatrix} + \begin{bmatrix}F_E\\ \hat{F} \end{bmatrix}\theta \right) \end{aligned}$$

(D3)

$$\begin{aligned} \begin{bmatrix}A_E\\ \hat{A}\end{bmatrix}^T\lambda&= -H\theta -c \end{aligned}$$

(D4)

$$\begin{aligned} \lambda (\theta )&= -\begin{bmatrix}A_E\\ \hat{A}\end{bmatrix}^{-T}\left( H\theta + c\right) \end{aligned}$$

(D5)

The extent of a particular critical region is based on remaining primal, dual, and parameter feasibility. This is shown in Eq. D6.

$$\begin{aligned} \text {CR} := \left\{ \theta \in \mathbb {R}^k:\quad \begin{matrix}Ax(\theta ) \le b + F\theta \\ \\ \lambda (\theta ) \ge 0 \\ \\ A_\theta \theta \le b_\theta \end{matrix}\right\} \end{aligned}$$

(D6)

Due to the affine nature of $x(\theta )$ and $\lambda (\theta )$ for mpQP and mpLP, the extent of a critical region is a polytope. As a post-processing step the redundant constraints of CR are removed via constraint elimination techniques.

Appendix E: Largest largest optimal active set detection

The objective of the optimization problem described in Eq. E1 gives an upper bound on the number of active constraints that can occur simultaneously at any optimal active set in the parametric space.

$$\begin{aligned} \max _{x,\theta , \lambda ,\mu , s, y} \quad \quad&\sum _i y_i \end{aligned}$$

(E1a)

$$\begin{aligned} \text {s.t. }\quad Qx + H\theta +&c + A_E^T\mu + A^T\lambda = 0 \end{aligned}$$

(E1b)

$$\begin{aligned} A_Ex&= b_E+F_E\theta \end{aligned}$$

(E1c)

$$\begin{aligned} Ax +s&= b+ F\theta \end{aligned}$$

(E1d)

$$\begin{aligned} 0\le&\lambda _i \le M^{\lambda _i}y_i \end{aligned}$$

(E1e)

$$\begin{aligned} 0\le&s_i \le M^{s_i}(1-y_i) \end{aligned}$$

(E1f)

$$\begin{aligned} A_\theta \theta&\le b_\theta \end{aligned}$$

(E1g)

$$\begin{aligned} x\in \mathbb {R}^m, \theta \in \mathbb {R}^k,&s\in \mathbb {R}^{t}, \lambda \in \mathbb {R}^{t}, \mu \in \mathbb {R}^t, y\in \{0,1\}^t \end{aligned}$$

(E1h)