An algorithm for solving the indefinite least squares problem with equality constraints

Nicola Mastronardi; Paul Van Dooren

doi:10.1007/s10543-013-0452-2

BIT Numerical Mathematics

March 2014, Volume 54, Issue 1, pp 201–218 | Cite as

An algorithm for solving the indefinite least squares problem with equality constraints

Authors
Authors and affiliations

Nicola MastronardiEmail author
Paul Van Dooren

Article

First Online: 11 October 2013

Received: 31 January 2013

Accepted: 27 September 2013

326 Downloads
6 Citations

Abstract

An algorithm for computing the solution of indefinite least squares problems and of indefinite least squares problems with equality constrained is presented. Such problems arise when solving total least squares problems and in H ^∞-smoothing.

The proposed algorithm relies only on stable orthogonal transformations reducing recursively the associated augmented matrix to proper block anti-triangular form. Some numerical results are reported showing the properties of the algorithm.

Keywords

Indefinite matrix Indefinite least squares Equality constraints

Mathematics Subject Classification (2010)

65F20 65G05 15A06

This is a preview of subscription content, to check access.

Appendix

We now describe how the linear system (3.9) can be solved when q<n−s.

Let s partition A ₁, ${\bf s}$ and $\tilde{{\bf b}}$ as follows,

$${A_1}= \left [ \begin{array}{c} {A}_{11} \\ {A}_{12} \end{array} \right ] \begin{array}{@{}l@{}} \left\}p, \right. \\ \left\} q, \right. \end{array} \qquad {\bf s}= \left [ \begin{array}{c} {\bf s}_{1} \\ {\bf s}_{2} \\ {\bf s}_3 \end{array} \right ] \begin{array}{@{}l@{}} \left\}n-s, \right. \\ \left\}p -n+s, \right. \\ \left\} q, \right. \end{array} \qquad \tilde{{\bf b}}= \left [ \begin{array}{c} \tilde{{\bf b}}_{1} \\ \tilde{{\bf b}}_{2} \\ \tilde{{\bf b}}_{3} \end{array} \right ] \begin{array}{@{}l@{}} \left\}n-s, \right. \\ \left\}p -n+s,\right. \\ \left\}q. \right. \end{array} $$

Compute the upper anti-QR factorization of A ₁₁, $[{Q}_{11}, {R}_{11}]={\tt aqru}({A}_{11})$ and A ₁₂, $[{Q}_{12}, {R}_{12}]={\tt aqru}({A}_{12})$.

Let

$$\tilde{Q}^{(2)} = \left [ \begin{array}{c@{\quad}c} {Q}_{11} & \\ & {Q}_{12} \end{array} \right ] \begin{array}{@{}l@{}} \left\} p \right. \\ \left\} q \right. \end{array} \quad \mbox{and} \quad Q^{(2)} = \left [ \begin{array}{c@{\quad}c} \tilde{Q}^{(2)} & \\ & I_{n-s} \end{array} \right ]. $$

Then (3.9) is transformed into the equivalent linear system

$$ M^{(3)}{{\bf y}}^{(3)}={{\bf f}}^{(3)}, $$

(6.1)

where

$$ {{\bf y}}^{(3)}= \left [ \begin{array}{@{}c@{}} {{\bf s}}^{(1)}_{1} \\ {{\bf s}}^{(1)}_{2} \\ {{\bf s}}^{(1)}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ] = Q^{(2)^T } \left [ \begin{array}{@{}c@{}} {{\bf s}}_{1} \\ {{\bf s}}_{2} \\ {{\bf s}}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ] = \left [ \begin{array}{@{}c@{}} {Q}_{11}^T \left [ \begin{array}{@{}c@{}} {\bf s}_{1} \\ {\bf s}_{2} \end{array} \right ] \\ {Q}_{12}^T {\bf s}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ], $$

(6.2)

$${\bf f}^{(3)} = \left [ \begin{array}{@{}c@{}} {\bf f}^{(3)}_{1} \\ {\bf f}^{(3)}_{2} \\ {\bf f}^{(3)}_3 \\ {\boldsymbol{0}} \end{array} \right ] = Q^{(2)^T } \left [ \begin{array}{@{}c@{}} \tilde{{\bf b}}_{1} \\ \tilde{{\bf b}}_{2} \\ \tilde{{\bf b}}_{3} \\ {\boldsymbol{0}} \end{array} \right ] = \left [ \begin{array}{@{}c@{}} {Q}_{11}^T \left [ \begin{array}{@{}c@{}} \tilde{{\bf b}}_1 \\ \tilde{{\bf b}}_2 \end{array} \right ] \\ {Q}_{12}^T \tilde{{\bf b}}_3 \\ {\boldsymbol{0}} \end{array} \right ]. $$

De to the structure of M ⁽³⁾, from (6.1) we can compute ${{\bf s}}^{(1)}_{2}$,

$$ {{\bf s}}^{(1)}_{2}= {\bf f}^{(3)}_{2}, $$

(6.3)

and “shrink” (6.1) to

$$ \tilde{M}^{(3)} \tilde{{\bf y}}^{(3)}= \tilde{{\bf f}}^{(3)}, $$

(6.4)

with

with $\operatorname{Inertia}(\tilde{M}^{(3)})= (q+n-s,0,p) - (0,0,p-n+s)=(q+n-s,0,n-s)$.

The matrix $\tilde{M}^{(3)}$ can be reduced to upper block antitriangular form by a sequence of n−s Householder transformations.

Let $\tilde{M}_{0}^{(3)}=\tilde{M}^{(3)}$.

At step i, i=1,…,q−1, the matrix $\tilde{M}_{i-1}^{(3)} = H_{i-1}\cdots H_{1} \tilde{M}_{0}^{(3)} H_{1}^{T} \cdots H_{i-1}^{T}$, is multiplied to the left by a Householder matrix $H_{i} \in{\mathbb{R}}^{(2(n-s)+q) \times(2(n-s)+q) } $ and to the right by the transpose of H _i, such that

$$\tilde{M}_{i}^{(3)} = H_{i}\tilde{M}_{i-1}^{(3)} H_{i}^T $$

has the rows (columns) i,n−s+1,n−s+2,…,n−s+i modified and the entries n−s+1,n−s+2,…,n−s+i in column (row) 2(n−s)+q−i+1 annihilated.

Furthermore, at step i, i=q,…,n−s, the matrix $\tilde{M}_{i-1}^{(3)} $ is multiplied to the left by a Householder matrix $H_{i} \in{\mathbb{R}}^{(2(n-s)+q) \times(2(n-s)+q) } $ and to the right by the transpose of H _i, such that

$$\tilde{M}_{i}^{(3)} = H_{i}\tilde{M}_{i-1}^{(3)} H_{i-1}^T $$

has the rows (columns) i,n−s+1,n−s+2,…,n−s+q modified and the entries n−s+1,n−s+2,…,n−s+q in column (row) 2(n−s)+q−i+1 annihilated.

Let $Q^{(3)} = H_{1}^{T} H_{2}^{T}\cdots H_{n-s}^{T} \in{\mathbb {R}}^{(2(n-s)+q) \times(2(n-s)+q)}$.

Then the linear system (6.4) is transformed into the equivalent one

$$ M^{(4)} {{\bf y}}^{(4)}= {{\bf f}}^{(4)}, $$

(6.5)

with M ⁽⁴⁾ having the following structure,

$$M^{(4)} = {Q^{(3)^T}}\tilde{M}^{(3)}{Q^{(3)}} = \left [ \begin{array}{c@{\quad}c@{\quad}c} {W^{(4)}} & {Z^{(4)}} & Y^{(4)} \\ {Z^{(4)}}^T & {X^{(4)}} & \\ {Y^{(4)}}^T & & \end{array} \right ] \begin{array}{@{}l@{}} \left\} n-s, \right. \\ \left\} q, \right. \\ \left\} n-s, \right. \end{array} $$

$Y^{(4)} \in{\mathbb{R}}^{(n-s) \times(n-s)}$ nonsingular upper anti-triangular, ${X^{(4)}}, {W^{(4)}}\in{\mathbb{R}}^{(n-s) \times (n-s)}$ symmetric, and

$$ {\bf y}^{(4)}= \left [ \begin{array}{c} {\bf y}^{(4)}_{1} \\ {\bf y}^{(4)}_{2} \\ {\bf y}^{(4)}_{3} \end{array} \right ] = {Q^{(3)^T}}\tilde{{\bf y}}^{(3)} = {Q^{(3)^T}} \left [ \begin{array}{c} {{\bf s}}^{(1)}_{1} \\ {{\bf s}}^{(1)}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ], $$

(6.6)

$${{\bf f}}^{(4)} = \left [ \begin{array}{c} {\bf f}^{(4)}_{1} \\ {\bf f}^{(4)}_{2} \\ {\bf f}^{(4)}_{3} \end{array} \right ] = {Q^{(3)^T}}\tilde{{\bf f}}^{(3)}= {Q^{(3)^T}} \left [ \begin{array}{c} {{\bf f}}^{(3)}_{1} \\ {{\bf f}}^{(3)}_3 \\ {\boldsymbol{0}} \end{array} \right ]. $$

Observe that ${\bf y}^{(4)}_{3}= \tilde{{\bf x}}_{1} $ and ${\bf f}^{(4)}_{3} = {\boldsymbol{0}}$, because of the structure of Q ⁽³⁾. Since $\operatorname{Inertia}(M^{(4)})=\operatorname{Inertia}(\tilde {M}^{(3)})=(q+n-s,0,n-s)$, then, by [6], the submatrix X ⁽⁴⁾ of M ⁽⁴⁾ is symmetric negative definite with Cholesky factorization $X^{(4)} = -L^{(4)} L^{(4)^{T}}$, $L^{(4)} \in{\mathbb{R}}^{q \times q}$ nonsingular lower triangular.

We can now solve the linear system (3.17) in the following steps.

Observe that ${\bf y}_{1}^{(4)} = {\boldsymbol{0}}$, since ${\bf f}^{(4)}_{3}= {\boldsymbol{0}}$. Therefore the ${\bf y}_{1}^{(4)}={\boldsymbol{0}}$ is the solution of the upper anti-triangular linear system
$$Y^{(4)^T} {\bf y}_1^{(4)}={\bf f}_3^{(4)}; $$
update the right-hand-side
$$\left [ \begin{array}{c} \tilde{{\bf f}}^{(4)}_1 \\ \tilde{{\bf f}}^{(4)}_2 \end{array} \right ]= \left [ \begin{array}{c} {{\bf f}}^{(4)}_1 \\ {{\bf f}}^{(4)}_2 \end{array} \right ]- \left [ \begin{array}{c} W^{(4)} \\ {Y^{(4)}}^T \end{array} \right ] {\bf y}_1^{(4)}; $$
solve the linear system $X^{(4)} {\bf y}_{2}^{(4)}= \tilde{{\bf f}}^{(4)}_{2} $,
$$\begin{aligned} &L^{(4)} {\bf t}= -\tilde{{\bf f}}^{(4)}_2 \\ &L^{(4)^T} {\bf y}_2^{(4)}= {\bf t}; \end{aligned}$$
solve the upper anti-triangular linear system
$${Y^{(4)}} {\bf y}^{(4)}_{3}={Y^{(4)}} \tilde{{\bf x}}_1=\tilde{{\bf f}}^{(4)}_1 -Z^{(4)}{\bf y}_2^{(4)}. $$

Once $\tilde{{\bf x}}_{1} $ is computed, the solution ${\bf x} $ of the problem (1.4) can be obtained as

$${\bf x}= \tilde{Q}^{(1)} \left [ \begin{array}{c} \tilde{{\bf x}}_1 \\ \tilde{{\bf x}}_2 \end{array} \right ]. $$

If one is also interested in the computation of the solution of the augmented system (1.7), from (6.6) Open image in new window

can be computed and, therefore, ${{\bf y}}^{(3)}$, since, by (6.3), ${{\bf s}}^{(1)}_{2}$ is already computed. Finally, from (6.2), ${\bf y}^{(2)}= Q^{(2)}{\bf y}^{(3)}$ can be computed.

About the computational complexity of this step, the computation of R ₁₁ and R ₁₂ requires 2(n−s)²(p−(n−s)/3) and 2q ²(q+n−s) floating point operations, respectively. Moreover, the computation of Y ⁽⁴⁾ requires 4q(n−s−q)²+2q ²(n−s)−2/3q ³ floating point operations.

References

1.
Bojanczyk, A., Higham, N.J., Patel, H.: The equality constrained indefinite least squares problem: theory and algorithms. BIT Numer. Math. 41(3), 505–517 (2003) CrossRefMathSciNetGoogle Scholar
2.
Bojanczyk, A., Higham, N.J., Patel, H.: Solving the indefinite least squares problem by hyperbolic QR factorization. SIAM J. Matrix Anal. Appl. 24(4), 914–931 (2003) CrossRefMATHMathSciNetGoogle Scholar
3.
Chandrasekaran, S., Gu, M., Sayed, A.H.: A stable and efficient algorithm for the indefinite linear least squares problem. SIAM J. Matrix Anal. Appl. 20, 354–362 (1998) CrossRefMATHMathSciNetGoogle Scholar
4.
Golub, G.H., Van Loan, C.F.: An analysis of the total least squares problem. SIAM J. Numer. Anal. 17, 883–893 (1980) CrossRefMATHMathSciNetGoogle Scholar
5.
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996) MATHGoogle Scholar
6.
Gould, N.I.M.: On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem. Math. Program. 32, 90–99 (1985) CrossRefMATHGoogle Scholar
7.
Hassibi, B., Sayed, A.H., Kailath, T.: Recursive linear estimation in Krein spaces–part I: theory. IEEE Trans. Autom. Control 41, 18–33 (1996) CrossRefMATHMathSciNetGoogle Scholar
8.
Higham, N.J.: J-Orthogonal matrices: properties and generation. SIAM Rev. 45(3), 504–519 (2003) CrossRefMATHMathSciNetGoogle Scholar
9.
Li, Q., Wang, M.: Algebraic properties and perturbation results for the indefinite least squares problem with equality constraints. Int. J. Comput. Math. 87(2), 425–434 (2010) CrossRefMathSciNetGoogle Scholar
10.
Li, Q., Pan, B.Z., Wang, Q.: The hyperbolic elimination method for solving the equality constrained indefinite least squares problem. Int. J. Comput. Math. 87(13), 2953–2966 (2010) CrossRefMathSciNetGoogle Scholar
11.
Mastronardi, N., Van Dooren, P.: The anti–triangular factorization of symmetric matrices. SIAM J. Matrix Anal. Appl. 34(1), 173–196 (2013) CrossRefMATHMathSciNetGoogle Scholar
12.
Mastronardi, N., Van Dooren, P.: Recursive approximation of the dominant eigenspace of an indefinite matrix. J. Comput. Appl. Math. 236(16), 4090–4104 (2012) CrossRefMATHMathSciNetGoogle Scholar
13.
Mastronardi, N., Van Dooren, P.: A structurally backward stable algorithm for solving the indefinite least squares problem with equality constraints. IMA J. Numer. Anal. (submitted) Google Scholar
14.
Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. SIAM, Philadelphia (1991) CrossRefMATHGoogle Scholar

Copyright information

Authors and Affiliations

Nicola Mastronardi
- 1
Email author
Paul Van Dooren
- 2

1.Istituto per le Applicazioni del Calcolo “M. Picone”Consiglio Nazionale delle Ricerche, sede di BariBariItaly
2.Department of Mathematical EngineeringCatholic University of LouvainLouvain-la-NeuveBelgium

An algorithm for solving the indefinite least squares problem with equality constraints

Abstract

Keywords

Mathematics Subject Classification (2010)

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite article