A fast eigenvalue approach for solving the trust region subproblem with an additional linear inequality

Abstract

In this paper, we study the extended trust region subproblem (eTRS) in which the trust region intersects the Euclidean ball with a single linear inequality constraint. By reformulating the Lagrangian dual of eTRS as a two-parameter linear eigenvalue problem, we state a necessary and sufficient condition for its strong duality in terms of an optimal solution of a linearly constrained bivariate concave maximization problem. This results in an efficient algorithm for solving eTRS of large size whenever the strong duality is detected. Finally, some numerical experiments are given to show the effectiveness of the proposed method.

Appendix

Proposition 1

Let $(t^{*},\lambda ^{*})$ be an optimal solution of problem (NewD-eTRS) with $\lambda _{\text {min}}(D(t^{*},\lambda ^{*}))=\lambda _{\text {min}}(A)$. Moreover, let $y(t^{*},\lambda ^{*})=[y_0(t^{*},\lambda ^{*}), z_0(t^{*},\lambda ^{*})^T]^T$ be a normalized eigenvector for $\lambda _{\text {min}}(D(t^{*},\lambda ^{*}))$ with $y_0(t^{*},\lambda ^{*})\ne 0$. Then $||x^{*}||^2\le \Delta$ where $x^{*}=\frac{z(t^{*},\lambda ^{*})}{y_0(t^{*},\lambda ^{*})}$.

Proof

First notice that, as discussed in proof of Item 1 of Theorem 4, we have

$$\begin{aligned} ||x^{*}||^2=\frac{1-y_0(t^{*},\lambda ^{*})^2}{y_0(t^{*},\lambda ^{*})^2}. \end{aligned}$$

On the other hand, since $(t^{*},\lambda ^{*})$ is an optimal solution of problem (NewD-eTRS), we have

$$\begin{aligned} t^{*}\in \text {argmax}_t\quad k(t,\lambda ^{*}):= (\Delta +1)\lambda _{\text {min}}(D(t,\lambda ^{*}))-t -\lambda ^{*} c. \end{aligned}$$

Moreover, we know, as shown in proof of Lemma 2, $t^{*}=t_0$ where $t_0$ is defined as before. The function $k(t,\lambda ^{*})$ is not differentiable at $t_0$ and the directional derivative from left, $k'(t_0^{-})=(\Delta +1)y_0(t^{*},\lambda ^{*})^2-1$. The optimality of $t_0$ implies that $k'(t_0^{-})\ge 0$. This proves that $||x^{*}||^2\le \Delta$.

Proposition 2

Suppose that $\lambda _{\text {min}}(A)$ has multiplicity, $i>2$ and let $\lambda _1^{*}:=-\lambda _{\text {min}}(A)\ge 0$ and $\lambda _2^{*}\ge 0$ be optimal Lagrange multipliers corresponding to the norm and the linear constraint of eTRS, respectively. Then $\lambda _1^{*}$ and $\lambda _2^{*}$ are also the optimal Lagrange multipliers corresponding to the norm and the linear constraint of eTRS with A replaced by $A+\alpha vv^T$ where v is a normalized eigenvector for $\lambda _{\text {min}}(A)$ such that $v^Tb=0$ and $\alpha >0$.

Proof

Let v be a normalized eigenvector for $\lambda _{\text {min}}(A)$ such that $v^Tb=0$. Moreover, let $A=PDP^T$ be the eigenvalue decomposition of A in which P contains v. First, since $\lambda _1^{*}=-\lambda _{\text {min}}(A)$ and $\lambda _2^{*}$ are the optimal Lagrange multipliers for eTRS, we have $(a-\frac{\lambda _2^{*}}{2}b)\in \text {Range}(A+\lambda _1^{*} I).$ This implies that $v^T(a-\frac{\lambda _2^{*}}{2}b)=0$ which results in $v^Ta=0$. As strong duality holds for eTRS, we know that $\lambda _1^{*}$ and $\lambda _2^{*}$ are the optimal solution of the Lagrangian dual of eTRS. Hence, to prove the statement, it is enough to show that the Lagrangian dual of eTRS is equivalent to the one for eTRS with A replaced by $A+\alpha vv^T$ where $\alpha$ is a positive constant. The Lagrangian dual of eTRS is given by

$$\begin{aligned} \max \quad&-(a-\frac{\lambda _2}{2}b)^T(A+\lambda _1 I )^{\dag }(a-\frac{\lambda _2}{2}b)-\lambda _1\Delta -\lambda _2 c\nonumber \\&\quad (a-\frac{\lambda _2}{2}b)\in \text {Range}(A+\lambda _1 I),\\&\quad \quad A+\lambda _1 I\succeq 0, \nonumber \\&\quad \quad \lambda _1\ge , \lambda _2\ge 0.\nonumber \end{aligned}$$

(15)

Now let $\lambda _1$ and $\lambda _2$ be feasible for problem (15). Then it is easy to verify the following facts:

$$\begin{aligned}&A+\lambda _1 I \succeq 0 \Leftrightarrow A+\alpha vv^T+\lambda _1 I \succeq 0,\end{aligned}$$

(16)

$$\begin{aligned}&(a-\frac{\lambda _2}{2}b)\in \text {Range}(A+\lambda _1 I)\Leftrightarrow (a-\frac{\lambda _2}{2}b)\in \text {Range}(A+\alpha vv^T+\lambda _1 I),\end{aligned}$$

(17)

$$\begin{aligned}&(A+\lambda _1 I )^{\dag }(a-\frac{\lambda _2}{2}b)=(A+\alpha vv^T+\lambda _1 I )^{\dag }(a-\frac{\lambda _2}{2}b). \end{aligned}$$

(18)

Equation (17) uses the fact that $v^Ta=v^Tb=0$. It follows from (15),(16) and (17) that $\lambda _1^{*}$ and $\lambda _2^{*}$ are also the optimal Lagrange multipliers for eTRS with $A+\alpha vv^T$. $\square$