Active-set prediction for interior point methods using controlled perturbations

Abstract

We propose the use of controlled perturbations to address the challenging question of optimal active-set prediction for interior point methods. Namely, in the context of linear programming, we consider perturbing the inequality constraints/bounds so as to enlarge the feasible set. We show that if the perturbations are chosen appropriately, the solution of the original problem lies on or close to the central path of the perturbed problem. We also find that a primal-dual path-following algorithm applied to the perturbed problem is able to accurately predict the optimal active set of the original problem when the duality gap for the perturbed problem is not too small; furthermore, depending on problem conditioning, this prediction can happen sooner than predicting the active set for the perturbed problem or when the original one is solved. Encouraging preliminary numerical experience is reported when comparing activity prediction for the perturbed and unperturbed problem formulations.

Appendices

Appendix 1: Proof of Lemma 2

An error bound for an optimization problem bounds the distance from a given point to the solution set in terms of a residual function [33]. In this section, we first formulate an lp problem as a monotone Linear Complementarity Problem (lcp) and then apply a global error bound for the monotone lcp to the reformulated lp problem in order to derive an error bound for the lp, and so prove Lemma 2.

By setting $s = c-A^{\top }y$ and $y=y^{+}-y^{-}$, where $y^{+}=\max (y,0)$ and $y^{-}=-\min (y,0)$, the first order optimality conditions (2) with $\lambda = 0$ for (PD) can be reformulated as

$$\begin{aligned} \begin{array}{r} \displaystyle Ax-b \ge 0,\,\, -Ax+b \ge 0,\\ \displaystyle c- A^{\top }y^{+}+A^{\top }y^{-} \ge 0, \\ \displaystyle x^{T}\left( c- A^{\top }y^{+}+A^{\top }y^{-}\right) = 0,\\ \displaystyle x \ge 0,\,\, y^{+} \ge 0,\,\, y^{-} \ge 0.\\ \end{array} \end{aligned}$$

(55)

Let

$$\begin{aligned} M= \begin{bmatrix} 0&-A^{T}&A^{T}\\ A&0&0\\ -A&0&0 \end{bmatrix}, \quad z= \begin{bmatrix} x\\ y^{+}\\ y^{-} \end{bmatrix} \quad \text{ and }\quad q= \begin{bmatrix} c\\ -b\\ b \end{bmatrix}, \end{aligned}$$

(56)

where A, b and c are (PD) problem data and $(x,y,s)\in {\mathbb {R}}^{n} \times {\mathbb {R}}^{m} \times {\mathbb {R}}^{n}$. Then finding a solution of (55) is equivalent to solving the following problem,

$$\begin{aligned} \quad Mz+q \ge 0, \quad z \ge 0, \quad z^{T}(Mz+q)=0,\quad \end{aligned}$$

(57)

where M, q and z are defined in (56), and z is considered to be the vector of variables.

Lemma 8

(PD) is equivalent to the lcp in (57) with M and q defined in (56), namely,

• 1. If $(x,y^{+},y^{-})$ is a solution of the lcp (57), then $(x,y,s)$ is a (PD) solution, where $y=y^{+}-y^{-}$ and $s=c-A^{T}y$.

• 2. If $(x,y,s)$ is a (PD) solution, then $(x,y^{+},y^{-})$ is a solution of the lcp (57).

Next we show that our lp problem can be viewed as a monotone lcp [7].

Lemma 9

The matrix M, defined in (56), is positive semidefinite, and so (57) is a monotone lcp.

Proof

For all $v=(v_{1},v_{2},v_{3})$, where $v_{1} \in {\mathbb {R}}^{n}, v_{2} \in {\mathbb {R}}^{m}$ and $v_{3} \in {\mathbb {R}}^{m}$, $v^{T} M v= v^{T}_{2}Av_{1} - v^{T}_{3}Av_{1} - v_{1}^{T}A^{T}v_{2} + v^{T}_{1}A^{T}v_{3} = 0$, since $v^{T}_{2}Av_{1} = (v^{T}_{2}Av_{1})^{T}=v^{T}_{1}A^{T}v_{2}$ and $v^{T}_{3}Av_{1}=(v^{T}_{3}Av_{1})^{T}=v^{T}_{1}A^{T}v_{3}$. Thus M is positive semidefinite. $\square$

A global error bound for a monotone lcp [24] is given next.

Lemma 10

(Mangasarian and Ren [24, Corollary 2.2]) Let z be any point away from the solution set of the monotone lcp (57) and $z^{*}$ be the closest solution of (57) to z under the Euclidean norm $\Vert \cdot \Vert$. Then $r(z)+w(z)$ is a global error bound for (57), namely,

$$\begin{aligned} \Vert z-z^{*}\Vert \le \tau (r(z)+w(z)), \end{aligned}$$

where $\tau$ is some problem-dependent constant, independent of z and $z^{*}$, and

$$\begin{aligned} \displaystyle \begin{array}{c} r(z)=\Vert z-(z-Mz-q)_{+}\Vert \\ \text {and}\\ w(z)=\left\| \left( -Mz-q,-z,z^{T}(Mz+q) \right) _{+}\right\| . \end{array} \end{aligned}$$

(58)

Lemma 11

Given the monotone lcp (57) with M and q defined in (56), let $(x,y^{+},y^{-})$ be any point away from the solution set of this problem and $(x^{*},(y^{*})^{+}$, $(y^{*})^{-})$ be the closest solution of this lcp to $(x,y^{+},y^{-})$ under the Euclidean norm $\Vert \cdot \Vert$. Then we have

$$\begin{aligned} \Vert \left( x,y^{+},y^{-}\right) - \left( x^{*},\left( y^{*}\right) ^{+},\left( y^{*}\right) ^{-}\right) \Vert \le \tau \left( r\left( x,y^{+},y^{-}\right) +w\left( x,y^{+},y^{-}\right) \right) , \end{aligned}$$

where $\tau$ is some problem-dependent constant, independent of $(x,y^{+},y^{-})$ and of $(x^{*},(y^{*})^{+},(y^{*})^{-})$,

$$\begin{aligned} r(x,y^{+},y^{-}) = \left\| \left( \, \min \left\{ x,c-A^{T}y\right\} ,\,\, \min \left\{ y^{+},Ax-b\right\} ,\,\, \min \left\{ y^{-}, b-Ax )\right\} \, \right) \right\| , \end{aligned}$$

(59)

and

$$\begin{aligned} w\left( x,y^{+},y^{-}\right) = \Vert \left( -\left( c-A^{T}y\right) ,b-Ax,\,Ax-b,\,-x,\, -y^{+},\, -y^{-},\,c^{T}x - b^{T}y\right) _{+}\Vert , \end{aligned}$$

(60)

and where $y = y^{+} - y^{-}$.

Proof

Substituting (56) into (58) and noting that $u - (u-v)_{+} = \min \left\{ u,v\right\}$ for any u, v vectors, we have

$$\begin{aligned} {\small { \begin{array}{l} r(x,y^{+},y^{-}) \\ \quad = \left\| \left( x - \left( x - \left( c - A^{T}\left( y^{+} - y^{-}\right) \right) \right) _{+},\,\, y^{+} - \left( y^{+} - (Ax - b)\right) _{+}, \,\, y^{-} - \left( y^{-} - ( b-Ax )\right) _{+} \right) \right\| , \end{array}}} \end{aligned}$$

and

$$\begin{aligned} {\small {\begin{array}{l} w\left( x,y^{+},y^{-}\right) \\ \quad = \left\| \left( -\left( c-A^{T} \left( y^{+} - y^{-} \right) \right) ,\,\,b-Ax,\,\,Ax-b,\,\,-x,\,\, -y^{+},\,\ -y^{-},\,\,c^{T}x - b^{T} \left( y^{+} - y^{-} \right) \right) _{+}\right\| . \end{array}}} \end{aligned}$$

Recalling $y = y^{+} - y^{-}$, (59) and (60) follow directly from the above equations. $\square$

Theorem 9

(Error bound for lp) Let $(x,y,s)\in {\mathbb {R}}^{n}\times {\mathbb {R}}^{m}\times {\mathbb {R}}^{n}$ where $s = c - A^{T}y$. Then there exist a (PD) solution $(x^{*},y^{*},s^{*})$

and problem-dependent constants $\tau _{p}$ and $\tau _{d}$, independent of $(x,y,s)$ and $\left( x^{*},y^{*},s^{*} \right)$, such that

$$\begin{aligned} \Vert x-x^{*}\Vert \le \tau _{p}\left( r(x,y) + w(x,y) \right) \quad \text {and} \quad \Vert s-s^{*}\Vert \le \tau _{d} \left( r(x,y) + w(x,y) \right) , \end{aligned}$$

where

$$\begin{aligned} r(x,y) = \left\| \left( \min \left\{ x,s\right\} ,\,\, \min \left\{ y^{+},Ax-b\right\} ,\,\, \min \left\{ y^{-},-Ax + b\right\} \right) \right\| , \end{aligned}$$

(61)

and

$$\begin{aligned} w(x,y) = \Vert (-s,\,\,b-Ax,\,\,Ax-b,\,\,-x,\,\,c^{T}x - b^{T}y)_{+}\Vert , \end{aligned}$$

(62)

and where $y^{+} = \max \left\{ y,0\right\}$ and $y^{-} = -\min \left\{ y,0\right\}$.

Proof

Consider the monotone lcp (57) with M and q defined in (56) and $z=(x,y^{+},y^{-})$. Let $z^{*}=(x^{*},(y^{*})^{+},(y^{*})^{-})$ be the closest solution to z in the solution set of this lcp. From Lemma 8, $(x^{*},y^{*},s^{*})$ with $y^{*}=(y^{*})^{+}-(y^{*})^{-}$ and $s^{*}=c-A^{T}y^{*}$ is a (PD) solution. (Note that we may lose the property that this is the closest solution to the given point.) From $( y^{+} ,y^{-} ) \ge 0$, $s = c- A^{T}y$ and Lemma 11, we have

$$\begin{aligned} \Vert \left( x,y^{+},y^{-}\right) - \left( x^{*},\left( y^{*}\right) ^{+},\left( y^{*}\right) ^{-}\right) \Vert \le \tau \left( r\left( x,s\right) +w\left( x,s\right) \right) , \end{aligned}$$

where r(x, s) and w(x, s) are defined in (61) and (62), respectively. This and norm properties give

$$\begin{aligned} \max \left( \Vert x-x^{*} \Vert , \Vert y^{+}- \left( y^{*}\right) ^{+} \Vert ,\Vert y^{-}- \left( y^{*}\right) ^{-} \Vert \right) \le \tau \left( r\left( x,s\right) +w\left( x,s\right) \right) , \end{aligned}$$

and so letting $\tau _{p} = \tau$, we deduce $\Vert x-x^{*} \Vert \le \tau _{p} (r(x,s)+w(x,s)).$ Since $s^{*} = c - A^{T} y^{*}$, we also have

$$\begin{aligned} \Vert s - s^{*}\Vert \le \Vert A^{T}\Vert \Vert y - y^{*}\Vert\le & {} \Vert A^{T}\Vert ( \Vert y^{+}- (y^{*})^{+} \Vert + \Vert y^{-}- (y^{*})^{-} \Vert )\\\le & {} \tau _{d} (r(x,s)+w(x,s)), \end{aligned}$$

where $\tau _{d}= 2\tau \Vert A\Vert$. $\square$

Proof of Lemma 2

Since $(x,y,s)\in {\mathcal {F}}^{0}_{\lambda }$, (1) gives $Ax=b$ and $A^{T}y+s=c$. Then the result follows directly from Theorem 9. $\square$

Appendix 2: Proof of Lemma 7

Theorem 3 shows that we are able to preserve the optimal strict complementarity partition after perturbing the problems if the original (PD) has a unique and nondegenerate solution. Actually, we can take a step further and show that then ($\hbox {PD}_{\lambda }$) will also have a unique and nondegenerate solution.

Theorem 10

Assume (6) holds and the (PD) problems have a unique and nondegenerate solution $\left( x^{*},y^{*},s^{*} \right)$. Let ${\mathcal {A}}$ and ${\mathcal {I}}$ denote the corresponding optimal active and inactive sets. Then there exists $\hat{\lambda }=\hat{\lambda }(A,b,c,x^{*},s^{*})>0$ such that the perturbed problems ($\hbox {PD}_{\lambda }$) with $0 \le \Vert \lambda \Vert < \hat{\lambda }$ have a unique and nondegenerate solution and the optimal active set is the same as that of the original (PD) problems.

Proof

We consider the equivalent perturbed problem (3). From Theorem 3, we know there exists a $\hat{\lambda }(A,b,c,x^{*},s^{*}) > 0$ such that (3) with $0 \le \Vert \lambda \Vert < \hat{\lambda }$ has a strictly complementary solution $\left( \hat{p},\hat{y},\hat{q} \right)$ with the same optimal active and inactive sets ${\mathcal {A}}$ and ${\mathcal {I}}$, namely we have

$$\begin{aligned} \hat{p}_{{\mathcal {I}}} > 0, \quad \hat{p}_{{\mathcal {A}}} = 0, \quad \hat{q}_{{\mathcal {A}}} > 0, \quad \text {and} \quad \hat{q}_{{\mathcal {I}}} = 0, \end{aligned}$$

and also

$$\begin{aligned} A_{{\mathcal {I}}}\hat{p}_{{\mathcal {I}}} = b_{\lambda }. \end{aligned}$$

(63)

Next we are about to show that $\left( \hat{p},\hat{y},\hat{q} \right)$ is the unique solution of (3). Assume there exists another solution $\bar{p} \ne \hat{p}$. Then $(\bar{p},\hat{y},\hat{q})$ satisfies the optimality conditions (4). From the complementarity equations (the third term) in (4) and $\hat{q}_{{\mathcal {A}}} > 0$ we have $\bar{p}_{{\mathcal {A}}} = 0 = \hat{p}_{{\mathcal {A}}}$. Then we have $A_{{\mathcal {I}}}\bar{p}_{{\mathcal {I}}} = b_{\lambda }.$ It follows from this and (63) that $A_{{\mathcal {I}}} ( \bar{p}_{{\mathcal {I}}} - \hat{p}_{{\mathcal {I}}} ) = 0.$ As the (PD) solution is unique and nondegenerate, we must have $|{\mathcal {I}}| = m$ and $rank(A_{{\mathcal {I}}}) = m$, namely, $A_{{\mathcal {I}}}$ is nonsingular, which implies $\hat{p}_{{\mathcal {I}}} = \bar{p}_{{\mathcal {I}}}$. Then (3) has a unique and nondegenerate primal solution, which also implies unique and nondegenerate dual solution. $\square$

To prove Lemma 7, we also need the following series of useful lemmas.

Lemma 12

(Farkas’ Lemma [2, Lemma 5.1]) One and only one of the following two systems has a solution:

$$\begin{aligned} \text{ System } \text{1: }&Tw \ge 0&\text{ and } \quad b^{T}w<0, \\ \text{ System } \text{2: }&T^{T}y=b&\text{ and } \quad y \ge 0, \end{aligned}$$

where $T \in {\mathbb {R}}^{m \times n}$, $b \in {\mathbb {R}}^{m}$, $w \in {\mathbb {R}}^{n}$ and $y\in {\mathbb {R}}^{m}$.

Lemma 13

Given $i \in \{1,\ldots ,n\}$, the following system

$$\begin{aligned} {\left\{ \begin{array}{ll} y+Ax \ge 0\\ x-A^{T}y \ge 0 \quad \text{ and } \quad x_{i} - A^{T}_{i}y > 0\\ (x,y) \ge 0 \end{array}\right. } \end{aligned}$$

always has a solution, where $A \in {\mathbb {R}}^{m \times n}$, $x \in {\mathbb {R}}^{n}$, $y\in {\mathbb {R}}^{m}$ and $A_{i}$ is the $i^{th}$ column of A.

Proof

Without losing generality, we can choose $i=1$. Partition x and A as $x = \left[ \, x_{1} \quad \bar{x}^{T} \,\right] ^{T}$ and $A = \begin{bmatrix} A_{1}&\bar{A} \end{bmatrix},$ where $\bar{x} = \left[ \, x_{2} \quad \dots \quad x_{n} \,\right] ^{T}$ and $\bar{A} = \begin{bmatrix} A_{2}&\dots&A_{n} \end{bmatrix}.$

We need to prove the following system has a solution

$$\begin{aligned} \left\{ \begin{array}{ccccccc} y &{} + &{} A_{1}x_{1} &{} + &{} \bar{A} \bar{x} &{} \ge &{} 0 \\ -\bar{A}^{T} y &{} &{} &{} + &{} \bar{x} &{} \ge &{} 0 \\ y &{} &{} &{} &{} &{} \ge &{} 0 \\ &{} &{} x_{1} &{} &{} &{} \ge &{} 0 \\ &{} &{} &{} &{} \bar{x} &{} \ge &{} 0 \\ A_{1}^{T} y &{} - &{} x_{1} &{} &{} &{} < &{} 0 \end{array} \right. . \end{aligned}$$

(64)

From Lemma 12, we know (64) has a solution if and only if

$$\begin{aligned} \left\{ \begin{array}{c} \begin{bmatrix} I_{m} &{} -\bar{A} &{} I_{m} &{} 0 &{} 0 \\ A^{T}_{1} &{} 0 &{} 0 &{} 1 &{} 0\\ \bar{A}^{T} &{} I_{n-1} &{} 0 &{} 0 &{} I_{n-1} \\ \end{bmatrix} \begin{bmatrix} u_{1} \\ u_{2} \\ u_{3} \\ u_{4} \\ u_{5} \end{bmatrix} = \begin{bmatrix} A_{1} \\ -1 \\ 0 \end{bmatrix} \\ (u_{1},u_{2},u_{3},u_{4},u_{5}) \ge 0 \end{array} \right. , \end{aligned}$$

(65)

has no solution, where $u_{1} \in {\mathbb {R}}^{m}$, $u_{2} \in {\mathbb {R}}^{n-1}$, $u_{3} \in {\mathbb {R}}^{m}$, $u_{4} \in {\mathbb {R}}$ and $u_{5} \in {\mathbb {R}}^{n-1}$.

Assume (65) has a solution $(u_{1},u_{2},u_{3},u_{4},u_{5}) \ge 0$. Then we get

$$\begin{aligned}&u_{1} - \bar{A}u_{2} - A_{1} = -u_{3} \le 0, \end{aligned}$$

(66a)

$$\begin{aligned}&A^{T}_{1}u_{1} = -1 - u_{4} <0, \end{aligned}$$

(66b)

$$\begin{aligned}&\bar{A}^{T} u_{1} = -u_{2} - u_{5} \le 0. \end{aligned}$$

(66c)

Multiplying both sides of (66a) by $u_{1}^{T} \ge 0$, we have

$$\begin{aligned} u^{T}_{1}u_{1} - (\bar{A}^{T}u_{1})^{T}u_{2} - A^{T}_{1}u_{1} = -u^{T}_{1}u_{3}. \end{aligned}$$

From (66b), (66c) and the nonnegativity of the variables, we know

$$\begin{aligned} u^{T}_{1}u_{1} - \left( \bar{A}^{T}u_{1}\right) ^{T}u_{2} - \bar{A}^{T}_{1}u_{1} > 0 \quad \text {but}\quad -u^{T}_{1}u_{3} \le 0. \end{aligned}$$

Thus (65) has no solution, which implies (64) has a solution. $\square$

Lemma 14

The system

$$\begin{aligned} {\left\{ \begin{array}{ll} y+Ax \ge 0\\ x-A^{T}y > 0\\ (x,y) \ge 0 \end{array}\right. } \end{aligned}$$

(67)

always has a solution, where $A \in {\mathbb {R}}^{m \times n}$, $x \in {\mathbb {R}}^{n}$ and $y\in {\mathbb {R}}^{m}$.

Proof

From Lemma 13, we know for any $i \in \{1,\ldots ,n\}$, there exists $(x^{i}, y^{i}) \ge 0$ where $x^{i} \in {\mathbb {R}}^{n}$ and $y^{i} \in {\mathbb {R}}^{m}$, such that

$$\begin{aligned} {\left\{ \begin{array}{ll} y^{i}+Ax^{i} \ge 0,\\ x^{i}-A^{T}y^{i} \ge 0 \quad \text{ and } \quad x^{i}_{i} - A^{T}_{i}y^{i} > 0.\\ \end{array}\right. } \end{aligned}$$

(68)

Set $\hat{x} = \sum _{i=1}^{n} x^{i} \ge 0$ and $\hat{y} = \sum _{i=1}^{n}y^{i} \ge 0$. Then from (68), we have

$$\begin{aligned} \hat{y} + A\hat{x} = \sum _{i=1}^{n}y^{i} + A\left( \sum _{i}^{n}x^{i}\right) = \sum _{i=1}^{n}\left( y^{i}+Ax^{i}\right) \ge 0, \end{aligned}$$

and

$$\begin{aligned} \hat{x} - A^{T}\hat{y} = \sum _{i=1}^{n}(x^{i} - A^{T}y^{i}) = \begin{bmatrix} \left( x^{1}_{1} - A_{1}^{T}y^{1}\right) + \left( x^{2}_{1} - A_{1}^{T}y^{2}\right) + \cdots + \left( x^{n}_{1} - A_{1}^{T}y^{n}\right) \\ \vdots \\ \left( x^{1}_{n} - A_{n}^{T}y^{1}\right) + \left( x^{2}_{n} - A_{n}^{T}y^{2}\right) + \cdots + \left( x^{n}_{n} - A_{n}^{T}y^{n}\right) \end{bmatrix} > 0. \end{aligned}$$

$\square$

Lemma 15

The system

$$\begin{aligned} {\left\{ \begin{array}{ll} y+Ax > 0\\ x-A^{T}y \ge 0\\ (x,y) \ge 0 \end{array}\right. } \end{aligned}$$

(69)

always has a solution, where $A \in {\mathbb {R}}^{m \times n}$, $x \in {\mathbb {R}}^{n}$ and $y\in {\mathbb {R}}^{m}$.

Proof

Replace A in Lemma 14 by $-A^{T}$. $\square$

Lemma 16

The system

$$\begin{aligned} {\left\{ \begin{array}{ll} y+Ax > 0\\ x-A^{T}y > 0\\ (x,y) \ge 0. \end{array}\right. } \end{aligned}$$

always has a solution, where $A \in {\mathbb {R}}^{m \times n}$, $x \in {\mathbb {R}}^{n}$ and $y\in {\mathbb {R}}^{m}$.

Proof

From Lemmas 14 and 15, we know there exist $(\hat{x}, \hat{y})$ and $(\tilde{x},\tilde{y})$ such that (67) and (69) hold respectively. Set $\bar{x} = \hat{x}+\tilde{x}$ and $\bar{y} = \hat{y}+\tilde{y}$ and deduce

$$\begin{aligned} \bar{y}+A\bar{x} = \underbrace{\left( \hat{y} + A \hat{x}\right) }_{\ge 0}+\underbrace{\left( \tilde{y}+A\tilde{x}\right) }_{>0} >0 \quad \text {and} \quad \bar{x} - A^{T}\bar{y} = \underbrace{\left( \hat{x} - A^{T} \hat{y}\right) }_{>0}+\underbrace{\left( \tilde{x}-A^{T}\tilde{y}\right) }_{\ge 0} >0. \end{aligned}$$

$\square$

Proof of Lemma 7

Assume ${\mathcal {A}}$ and ${\mathcal {I}}$ are the optimal active and inactive sets at the unique solution $\left( x^{*},y^{*},s^{*} \right)$ of (PD). Then from (45), we have

$$\begin{aligned} \epsilon (A,b,c) = \min \left( \min _{i \in {\mathcal {I}}}\left( x^{*}_{i}\right) ,\min _{i \in {\mathcal {A}}}\left( s^{*}_{i}\right) \right) . \end{aligned}$$

(70)

From Theorem 10, we know there exists a $\hat{\lambda } = \hat{\lambda } (A,b,c,x^{*},s^{*})$ such that ($\hbox {PD}_{\lambda }$) with $0 \le \Vert \lambda \Vert < \hat{\lambda }$ has a unique and nondegenerate solution and ${\mathcal {A}}$ and ${\mathcal {I}}$ are the optimal active and inactive sets. Since $\left( \hat{p},\hat{y},\hat{q} \right)$ defined in (13) is a solution of (3), $\left( x^{*}_{\lambda },y^{*}_{\lambda },s^{*}_{\lambda } \right) = ( \hat{p} - \lambda , \hat{y}, \hat{q} - \lambda )$ is a solution of ($\hbox {PD}_{\lambda }$) and also unique, with ${\mathcal {A}}$ and ${\mathcal {I}}$ being the optimal active and inactive sets. This and (21) give

$$\begin{aligned} \epsilon \left( A,b_{\lambda },c_{\lambda }\right) = \min \left( \min _{i \in {\mathcal {I}}}\left( \hat{x}_{i} + \lambda _{i}\right) ,\min _{i \in {\mathcal {A}}}\left( \hat{s}_{i} + \lambda _{i}\right) \right) . \end{aligned}$$

(71)

From (13), recalling that $\hat{p} = \hat{x} + \lambda$ and $\hat{q} = \hat{s} + \lambda$, we have

$$\begin{aligned} \left( \hat{x}_{{\mathcal {I}}} +\lambda _{{\mathcal {I}}} \right) - x^{*}_{{\mathcal {I}}} = \lambda _{{\mathcal {I}}} + A^{-1}_{{\mathcal {I}}}A_{{\mathcal {A}}}\lambda _{{\mathcal {A}}} \quad \text {and}\quad \left( \hat{s}_{{\mathcal {A}}} + \lambda _{{\mathcal {A}}} \right) - s^{*}_{{\mathcal {A}}} = \lambda _{{\mathcal {A}}} - \left( A_{{\mathcal {I}}}^{-1}A_{{\mathcal {A}}}\right) ^{T}\lambda _{{\mathcal {I}}}. \end{aligned}$$

This, (70) and (71) give us that $\epsilon (A,b_{\lambda },c_{\lambda }) > \epsilon (A,b,c)$, provided

$$\begin{aligned} {\left\{ \begin{array}{ll} \lambda _{{\mathcal {I}}}+A_{{\mathcal {I}}}^{-1}A_{{\mathcal {A}}}\lambda _{{\mathcal {A}}} > 0\\ \lambda _{{\mathcal {A}}} - (A_{{\mathcal {I}}}^{-1}A_{{\mathcal {A}}})^{T} \lambda _{{\mathcal {I}}} > 0\\ \lambda _{{\mathcal {I}}}, \lambda _{{\mathcal {A}}} \ge 0 \end{array}\right. }. \end{aligned}$$

(72)

It remains to find a solution of (72) whose norm is less than $\hat{\lambda }$. From Lemma 16, we know (72) always has a solution, say $\bar{\lambda }$. Since (72) is homogeneous, $\frac{\hat{\lambda }}{2 \Vert \bar{\lambda }\Vert }\bar{\lambda }$ is also a solution, and $\Vert \frac{\hat{\lambda }}{2 \Vert \bar{\lambda }\Vert }\bar{\lambda } \Vert < \hat{\lambda }$. Without losing generality, we denote this solution as $\bar{\lambda }$. Furthermore, (72) holds for all $\lambda$ with $0<\lambda = \alpha \bar{\lambda }<\bar{\lambda }$ where $\alpha \in (0,1)$. $\square$

Appendix 3: An active-set prediction procedure

Note that in Procedure 1, ${\mathcal {A}}^{k}$ is the predicted active set, ${\mathcal {I}}^{k}$, the predicted inactive set and ${\mathcal {Z}}^{k} = \{ 1,2,\ldots ,n \} \backslash \left( {{\mathcal {A}}}^{k} \cup {{\mathcal {I}}}^{k}\right)$, the set of all undetermined indices at the $k^{th}$ iteration.

Appendix 4: Results for crossover to simplex on selected Netlib problems

From the left to the right, we give the name of the test problems, number of equality constraints, number of variables, the value of duality gap $\mu _{\lambda }^{K}$ when we terminate the (perturbed) Algorithm 1, the value of duality gap $\mu ^{K}$ when we terminate the (unperturbed) Algorithm 2, number of ipm iterations, the relative difference (see Footnote 7 on Page 26) between two bases generated from Algorithms 1 and 2, simplex iterations for Algorithm 1 and the simplex iterations for Algorithm 2. Since the algorithm without perturbations is terminated at the same ipm iteration as Algorithm 1, we show only the number of ipm iterations for the latter. Problems on which Algorithm 1 loses are marked in bold font. ‘—’ means the simplex solver fails for a particular test problem.

Table 3 Crossover to simplex test on a selection of Netlib problems