An algorithm for solving the indefinite least squares problem with equality constraints

Abstract

An algorithm for computing the solution of indefinite least squares problems and of indefinite least squares problems with equality constrained is presented. Such problems arise when solving total least squares problems and in H ^∞-smoothing.

The proposed algorithm relies only on stable orthogonal transformations reducing recursively the associated augmented matrix to proper block anti-triangular form. Some numerical results are reported showing the properties of the algorithm.

Appendix

We now describe how the linear system (3.9) can be solved when q<n−s.

Let s partition A ₁, ${\bf s}$ and $\tilde{{\bf b}}$ as follows,

$${A_1}= \left [ \begin{array}{c} {A}_{11} \\ {A}_{12} \end{array} \right ] \begin{array}{@{}l@{}} \left\}p, \right. \\ \left\} q, \right. \end{array} \qquad {\bf s}= \left [ \begin{array}{c} {\bf s}_{1} \\ {\bf s}_{2} \\ {\bf s}_3 \end{array} \right ] \begin{array}{@{}l@{}} \left\}n-s, \right. \\ \left\}p -n+s, \right. \\ \left\} q, \right. \end{array} \qquad \tilde{{\bf b}}= \left [ \begin{array}{c} \tilde{{\bf b}}_{1} \\ \tilde{{\bf b}}_{2} \\ \tilde{{\bf b}}_{3} \end{array} \right ] \begin{array}{@{}l@{}} \left\}n-s, \right. \\ \left\}p -n+s,\right. \\ \left\}q. \right. \end{array} $$

Compute the upper anti-QR factorization of A ₁₁, $[{Q}_{11}, {R}_{11}]={\tt aqru}({A}_{11})$ and A ₁₂, $[{Q}_{12}, {R}_{12}]={\tt aqru}({A}_{12})$.

Let

$$\tilde{Q}^{(2)} = \left [ \begin{array}{c@{\quad}c} {Q}_{11} & \\ & {Q}_{12} \end{array} \right ] \begin{array}{@{}l@{}} \left\} p \right. \\ \left\} q \right. \end{array} \quad \mbox{and} \quad Q^{(2)} = \left [ \begin{array}{c@{\quad}c} \tilde{Q}^{(2)} & \\ & I_{n-s} \end{array} \right ]. $$

Then (3.9) is transformed into the equivalent linear system

$$ M^{(3)}{{\bf y}}^{(3)}={{\bf f}}^{(3)}, $$

(6.1)

where

$$ {{\bf y}}^{(3)}= \left [ \begin{array}{@{}c@{}} {{\bf s}}^{(1)}_{1} \\ {{\bf s}}^{(1)}_{2} \\ {{\bf s}}^{(1)}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ] = Q^{(2)^T } \left [ \begin{array}{@{}c@{}} {{\bf s}}_{1} \\ {{\bf s}}_{2} \\ {{\bf s}}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ] = \left [ \begin{array}{@{}c@{}} {Q}_{11}^T \left [ \begin{array}{@{}c@{}} {\bf s}_{1} \\ {\bf s}_{2} \end{array} \right ] \\ {Q}_{12}^T {\bf s}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ], $$

(6.2)

$${\bf f}^{(3)} = \left [ \begin{array}{@{}c@{}} {\bf f}^{(3)}_{1} \\ {\bf f}^{(3)}_{2} \\ {\bf f}^{(3)}_3 \\ {\boldsymbol{0}} \end{array} \right ] = Q^{(2)^T } \left [ \begin{array}{@{}c@{}} \tilde{{\bf b}}_{1} \\ \tilde{{\bf b}}_{2} \\ \tilde{{\bf b}}_{3} \\ {\boldsymbol{0}} \end{array} \right ] = \left [ \begin{array}{@{}c@{}} {Q}_{11}^T \left [ \begin{array}{@{}c@{}} \tilde{{\bf b}}_1 \\ \tilde{{\bf b}}_2 \end{array} \right ] \\ {Q}_{12}^T \tilde{{\bf b}}_3 \\ {\boldsymbol{0}} \end{array} \right ]. $$

De to the structure of M ⁽³⁾, from (6.1) we can compute ${{\bf s}}^{(1)}_{2}$,

$$ {{\bf s}}^{(1)}_{2}= {\bf f}^{(3)}_{2}, $$

(6.3)

and “shrink” (6.1) to

$$ \tilde{M}^{(3)} \tilde{{\bf y}}^{(3)}= \tilde{{\bf f}}^{(3)}, $$

(6.4)

with

with $\operatorname{Inertia}(\tilde{M}^{(3)})= (q+n-s,0,p) - (0,0,p-n+s)=(q+n-s,0,n-s)$.

The matrix $\tilde{M}^{(3)}$ can be reduced to upper block antitriangular form by a sequence of n−s Householder transformations.

Let $\tilde{M}_{0}^{(3)}=\tilde{M}^{(3)}$.

At step i, i=1,…,q−1, the matrix $\tilde{M}_{i-1}^{(3)} = H_{i-1}\cdots H_{1} \tilde{M}_{0}^{(3)} H_{1}^{T} \cdots H_{i-1}^{T}$, is multiplied to the left by a Householder matrix $H_{i} \in{\mathbb{R}}^{(2(n-s)+q) \times(2(n-s)+q) }$ and to the right by the transpose of H _i , such that

$$\tilde{M}_{i}^{(3)} = H_{i}\tilde{M}_{i-1}^{(3)} H_{i}^T $$

has the rows (columns) i,n−s+1,n−s+2,…,n−s+i modified and the entries n−s+1,n−s+2,…,n−s+i in column (row) 2(n−s)+q−i+1 annihilated.

Furthermore, at step i, i=q,…,n−s, the matrix $\tilde{M}_{i-1}^{(3)}$ is multiplied to the left by a Householder matrix $H_{i} \in{\mathbb{R}}^{(2(n-s)+q) \times(2(n-s)+q) }$ and to the right by the transpose of H _i , such that

$$\tilde{M}_{i}^{(3)} = H_{i}\tilde{M}_{i-1}^{(3)} H_{i-1}^T $$

has the rows (columns) i,n−s+1,n−s+2,…,n−s+q modified and the entries n−s+1,n−s+2,…,n−s+q in column (row) 2(n−s)+q−i+1 annihilated.

Let $Q^{(3)} = H_{1}^{T} H_{2}^{T}\cdots H_{n-s}^{T} \in{\mathbb {R}}^{(2(n-s)+q) \times(2(n-s)+q)}$.

Then the linear system (6.4) is transformed into the equivalent one

$$ M^{(4)} {{\bf y}}^{(4)}= {{\bf f}}^{(4)}, $$

(6.5)

with M ⁽⁴⁾ having the following structure,

$$M^{(4)} = {Q^{(3)^T}}\tilde{M}^{(3)}{Q^{(3)}} = \left [ \begin{array}{c@{\quad}c@{\quad}c} {W^{(4)}} & {Z^{(4)}} & Y^{(4)} \\ {Z^{(4)}}^T & {X^{(4)}} & \\ {Y^{(4)}}^T & & \end{array} \right ] \begin{array}{@{}l@{}} \left\} n-s, \right. \\ \left\} q, \right. \\ \left\} n-s, \right. \end{array} $$

$Y^{(4)} \in{\mathbb{R}}^{(n-s) \times(n-s)}$ nonsingular upper anti-triangular, ${X^{(4)}}, {W^{(4)}}\in{\mathbb{R}}^{(n-s) \times (n-s)}$ symmetric, and

$$ {\bf y}^{(4)}= \left [ \begin{array}{c} {\bf y}^{(4)}_{1} \\ {\bf y}^{(4)}_{2} \\ {\bf y}^{(4)}_{3} \end{array} \right ] = {Q^{(3)^T}}\tilde{{\bf y}}^{(3)} = {Q^{(3)^T}} \left [ \begin{array}{c} {{\bf s}}^{(1)}_{1} \\ {{\bf s}}^{(1)}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ], $$

(6.6)

$${{\bf f}}^{(4)} = \left [ \begin{array}{c} {\bf f}^{(4)}_{1} \\ {\bf f}^{(4)}_{2} \\ {\bf f}^{(4)}_{3} \end{array} \right ] = {Q^{(3)^T}}\tilde{{\bf f}}^{(3)}= {Q^{(3)^T}} \left [ \begin{array}{c} {{\bf f}}^{(3)}_{1} \\ {{\bf f}}^{(3)}_3 \\ {\boldsymbol{0}} \end{array} \right ]. $$

Observe that ${\bf y}^{(4)}_{3}= \tilde{{\bf x}}_{1}$ and ${\bf f}^{(4)}_{3} = {\boldsymbol{0}}$, because of the structure of Q ⁽³⁾. Since $\operatorname{Inertia}(M^{(4)})=\operatorname{Inertia}(\tilde {M}^{(3)})=(q+n-s,0,n-s)$, then, by [6], the submatrix X ⁽⁴⁾ of M ⁽⁴⁾ is symmetric negative definite with Cholesky factorization $X^{(4)} = -L^{(4)} L^{(4)^{T}}$, $L^{(4)} \in{\mathbb{R}}^{q \times q}$ nonsingular lower triangular.

We can now solve the linear system (3.17) in the following steps.

• Observe that ${\bf y}_{1}^{(4)} = {\boldsymbol{0}}$, since ${\bf f}^{(4)}_{3}= {\boldsymbol{0}}$. Therefore the ${\bf y}_{1}^{(4)}={\boldsymbol{0}}$ is the solution of the upper anti-triangular linear system

  $$Y^{(4)^T} {\bf y}_1^{(4)}={\bf f}_3^{(4)}; $$

• update the right-hand-side

  $$\left [ \begin{array}{c} \tilde{{\bf f}}^{(4)}_1 \\ \tilde{{\bf f}}^{(4)}_2 \end{array} \right ]= \left [ \begin{array}{c} {{\bf f}}^{(4)}_1 \\ {{\bf f}}^{(4)}_2 \end{array} \right ]- \left [ \begin{array}{c} W^{(4)} \\ {Y^{(4)}}^T \end{array} \right ] {\bf y}_1^{(4)}; $$

• solve the linear system $X^{(4)} {\bf y}_{2}^{(4)}= \tilde{{\bf f}}^{(4)}_{2}$,

  $$\begin{aligned} &L^{(4)} {\bf t}= -\tilde{{\bf f}}^{(4)}_2 \\ &L^{(4)^T} {\bf y}_2^{(4)}= {\bf t}; \end{aligned}$$

• solve the upper anti-triangular linear system

  $${Y^{(4)}} {\bf y}^{(4)}_{3}={Y^{(4)}} \tilde{{\bf x}}_1=\tilde{{\bf f}}^{(4)}_1 -Z^{(4)}{\bf y}_2^{(4)}. $$

Once $\tilde{{\bf x}}_{1}$ is computed, the solution ${\bf x}$ of the problem (1.4) can be obtained as

$${\bf x}= \tilde{Q}^{(1)} \left [ \begin{array}{c} \tilde{{\bf x}}_1 \\ \tilde{{\bf x}}_2 \end{array} \right ]. $$

If one is also interested in the computation of the solution of the augmented system (1.7), from (6.6) can be computed and, therefore, ${{\bf y}}^{(3)}$, since, by (6.3), ${{\bf s}}^{(1)}_{2}$ is already computed. Finally, from (6.2), ${\bf y}^{(2)}= Q^{(2)}{\bf y}^{(3)}$ can be computed.

About the computational complexity of this step, the computation of R ₁₁ and R ₁₂ requires 2(n−s)²(p−(n−s)/3) and 2q ²(q+n−s) floating point operations, respectively. Moreover, the computation of Y ⁽⁴⁾ requires 4q(n−s−q)²+2q ²(n−s)−2/3q ³ floating point operations.