An algorithm for solving the indefinite least squares problem with equality constraints

Abstract

An algorithm for computing the solution of indefinite least squares problems and of indefinite least squares problems with equality constrained is presented. Such problems arise when solving total least squares problems and in H -smoothing.

The proposed algorithm relies only on stable orthogonal transformations reducing recursively the associated augmented matrix to proper block anti-triangular form. Some numerical results are reported showing the properties of the algorithm.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    The condition number of a rectangular matrix \(A \in{\mathbb{R}}^{m \times n}\), mn, \(\operatorname{rank}(A)=n\), is defined as σ max(A)/σ min(A) [5].

References

  1. 1.

    Bojanczyk, A., Higham, N.J., Patel, H.: The equality constrained indefinite least squares problem: theory and algorithms. BIT Numer. Math. 41(3), 505–517 (2003)

    Article  MathSciNet  Google Scholar 

  2. 2.

    Bojanczyk, A., Higham, N.J., Patel, H.: Solving the indefinite least squares problem by hyperbolic QR factorization. SIAM J. Matrix Anal. Appl. 24(4), 914–931 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  3. 3.

    Chandrasekaran, S., Gu, M., Sayed, A.H.: A stable and efficient algorithm for the indefinite linear least squares problem. SIAM J. Matrix Anal. Appl. 20, 354–362 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  4. 4.

    Golub, G.H., Van Loan, C.F.: An analysis of the total least squares problem. SIAM J. Numer. Anal. 17, 883–893 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  5. 5.

    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996)

    Google Scholar 

  6. 6.

    Gould, N.I.M.: On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem. Math. Program. 32, 90–99 (1985)

    Article  MATH  Google Scholar 

  7. 7.

    Hassibi, B., Sayed, A.H., Kailath, T.: Recursive linear estimation in Krein spaces–part I: theory. IEEE Trans. Autom. Control 41, 18–33 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  8. 8.

    Higham, N.J.: J-Orthogonal matrices: properties and generation. SIAM Rev. 45(3), 504–519 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  9. 9.

    Li, Q., Wang, M.: Algebraic properties and perturbation results for the indefinite least squares problem with equality constraints. Int. J. Comput. Math. 87(2), 425–434 (2010)

    Article  MathSciNet  Google Scholar 

  10. 10.

    Li, Q., Pan, B.Z., Wang, Q.: The hyperbolic elimination method for solving the equality constrained indefinite least squares problem. Int. J. Comput. Math. 87(13), 2953–2966 (2010)

    Article  MathSciNet  Google Scholar 

  11. 11.

    Mastronardi, N., Van Dooren, P.: The anti–triangular factorization of symmetric matrices. SIAM J. Matrix Anal. Appl. 34(1), 173–196 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  12. 12.

    Mastronardi, N., Van Dooren, P.: Recursive approximation of the dominant eigenspace of an indefinite matrix. J. Comput. Appl. Math. 236(16), 4090–4104 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  13. 13.

    Mastronardi, N., Van Dooren, P.: A structurally backward stable algorithm for solving the indefinite least squares problem with equality constraints. IMA J. Numer. Anal. (submitted)

  14. 14.

    Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. SIAM, Philadelphia (1991)

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Nicola Mastronardi.

Additional information

The work of the first author is partly supported by the GNCS INdAM project “Strategie risolutive per sistemi lineari di tipo KKT con uso di informazioni strutturali”. The work of the second author is partly supported by the Belgian Network DYSCO (Dynamical Systems, Control, and Optimization), funded by the Interuniversity Attraction Poles Programme, initiated by the Belgian State, Science Policy Office. The scientific responsibility rests with its authors.

Communicated by Miloud Sadkane.

Appendix

Appendix

We now describe how the linear system (3.9) can be solved when q<ns.

Let s partition A 1, \({\bf s}\) and \(\tilde{{\bf b}}\) as follows,

$${A_1}= \left [ \begin{array}{c} {A}_{11} \\ {A}_{12} \end{array} \right ] \begin{array}{@{}l@{}} \left\}p, \right. \\ \left\} q, \right. \end{array} \qquad {\bf s}= \left [ \begin{array}{c} {\bf s}_{1} \\ {\bf s}_{2} \\ {\bf s}_3 \end{array} \right ] \begin{array}{@{}l@{}} \left\}n-s, \right. \\ \left\}p -n+s, \right. \\ \left\} q, \right. \end{array} \qquad \tilde{{\bf b}}= \left [ \begin{array}{c} \tilde{{\bf b}}_{1} \\ \tilde{{\bf b}}_{2} \\ \tilde{{\bf b}}_{3} \end{array} \right ] \begin{array}{@{}l@{}} \left\}n-s, \right. \\ \left\}p -n+s,\right. \\ \left\}q. \right. \end{array} $$

Compute the upper anti-QR factorization of A 11, \([{Q}_{11}, {R}_{11}]={\tt aqru}({A}_{11})\) and A 12, \([{Q}_{12}, {R}_{12}]={\tt aqru}({A}_{12})\).

Let

$$\tilde{Q}^{(2)} = \left [ \begin{array}{c@{\quad}c} {Q}_{11} & \\ & {Q}_{12} \end{array} \right ] \begin{array}{@{}l@{}} \left\} p \right. \\ \left\} q \right. \end{array} \quad \mbox{and} \quad Q^{(2)} = \left [ \begin{array}{c@{\quad}c} \tilde{Q}^{(2)} & \\ & I_{n-s} \end{array} \right ]. $$

Then (3.9) is transformed into the equivalent linear system

$$ M^{(3)}{{\bf y}}^{(3)}={{\bf f}}^{(3)}, $$
(6.1)

where

figureh
$$ {{\bf y}}^{(3)}= \left [ \begin{array}{@{}c@{}} {{\bf s}}^{(1)}_{1} \\ {{\bf s}}^{(1)}_{2} \\ {{\bf s}}^{(1)}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ] = Q^{(2)^T } \left [ \begin{array}{@{}c@{}} {{\bf s}}_{1} \\ {{\bf s}}_{2} \\ {{\bf s}}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ] = \left [ \begin{array}{@{}c@{}} {Q}_{11}^T \left [ \begin{array}{@{}c@{}} {\bf s}_{1} \\ {\bf s}_{2} \end{array} \right ] \\ {Q}_{12}^T {\bf s}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ], $$
(6.2)
$${\bf f}^{(3)} = \left [ \begin{array}{@{}c@{}} {\bf f}^{(3)}_{1} \\ {\bf f}^{(3)}_{2} \\ {\bf f}^{(3)}_3 \\ {\boldsymbol{0}} \end{array} \right ] = Q^{(2)^T } \left [ \begin{array}{@{}c@{}} \tilde{{\bf b}}_{1} \\ \tilde{{\bf b}}_{2} \\ \tilde{{\bf b}}_{3} \\ {\boldsymbol{0}} \end{array} \right ] = \left [ \begin{array}{@{}c@{}} {Q}_{11}^T \left [ \begin{array}{@{}c@{}} \tilde{{\bf b}}_1 \\ \tilde{{\bf b}}_2 \end{array} \right ] \\ {Q}_{12}^T \tilde{{\bf b}}_3 \\ {\boldsymbol{0}} \end{array} \right ]. $$

De to the structure of M (3), from (6.1) we can compute \({{\bf s}}^{(1)}_{2}\),

$$ {{\bf s}}^{(1)}_{2}= {\bf f}^{(3)}_{2}, $$
(6.3)

and “shrink” (6.1) to

$$ \tilde{M}^{(3)} \tilde{{\bf y}}^{(3)}= \tilde{{\bf f}}^{(3)}, $$
(6.4)

with

figurei

with \(\operatorname{Inertia}(\tilde{M}^{(3)})= (q+n-s,0,p) - (0,0,p-n+s)=(q+n-s,0,n-s)\).

The matrix \(\tilde{M}^{(3)}\) can be reduced to upper block antitriangular form by a sequence of ns Householder transformations.

Let \(\tilde{M}_{0}^{(3)}=\tilde{M}^{(3)}\).

At step i, i=1,…,q−1, the matrix \(\tilde{M}_{i-1}^{(3)} = H_{i-1}\cdots H_{1} \tilde{M}_{0}^{(3)} H_{1}^{T} \cdots H_{i-1}^{T}\), is multiplied to the left by a Householder matrix \(H_{i} \in{\mathbb{R}}^{(2(n-s)+q) \times(2(n-s)+q) } \) and to the right by the transpose of H i , such that

$$\tilde{M}_{i}^{(3)} = H_{i}\tilde{M}_{i-1}^{(3)} H_{i}^T $$

has the rows (columns) i,ns+1,ns+2,…,ns+i modified and the entries ns+1,ns+2,…,ns+i in column (row) 2(ns)+qi+1 annihilated.

Furthermore, at step i, i=q,…,ns, the matrix \(\tilde{M}_{i-1}^{(3)} \) is multiplied to the left by a Householder matrix \(H_{i} \in{\mathbb{R}}^{(2(n-s)+q) \times(2(n-s)+q) } \) and to the right by the transpose of H i , such that

$$\tilde{M}_{i}^{(3)} = H_{i}\tilde{M}_{i-1}^{(3)} H_{i-1}^T $$

has the rows (columns) i,ns+1,ns+2,…,ns+q modified and the entries ns+1,ns+2,…,ns+q in column (row) 2(ns)+qi+1 annihilated.

Let \(Q^{(3)} = H_{1}^{T} H_{2}^{T}\cdots H_{n-s}^{T} \in{\mathbb {R}}^{(2(n-s)+q) \times(2(n-s)+q)}\).

Then the linear system (6.4) is transformed into the equivalent one

$$ M^{(4)} {{\bf y}}^{(4)}= {{\bf f}}^{(4)}, $$
(6.5)

with M (4) having the following structure,

$$M^{(4)} = {Q^{(3)^T}}\tilde{M}^{(3)}{Q^{(3)}} = \left [ \begin{array}{c@{\quad}c@{\quad}c} {W^{(4)}} & {Z^{(4)}} & Y^{(4)} \\ {Z^{(4)}}^T & {X^{(4)}} & \\ {Y^{(4)}}^T & & \end{array} \right ] \begin{array}{@{}l@{}} \left\} n-s, \right. \\ \left\} q, \right. \\ \left\} n-s, \right. \end{array} $$

\(Y^{(4)} \in{\mathbb{R}}^{(n-s) \times(n-s)}\) nonsingular upper anti-triangular, \({X^{(4)}}, {W^{(4)}}\in{\mathbb{R}}^{(n-s) \times (n-s)}\) symmetric, and

$$ {\bf y}^{(4)}= \left [ \begin{array}{c} {\bf y}^{(4)}_{1} \\ {\bf y}^{(4)}_{2} \\ {\bf y}^{(4)}_{3} \end{array} \right ] = {Q^{(3)^T}}\tilde{{\bf y}}^{(3)} = {Q^{(3)^T}} \left [ \begin{array}{c} {{\bf s}}^{(1)}_{1} \\ {{\bf s}}^{(1)}_3 \\ \tilde{{\bf x}}_1 \end{array} \right ], $$
(6.6)
$${{\bf f}}^{(4)} = \left [ \begin{array}{c} {\bf f}^{(4)}_{1} \\ {\bf f}^{(4)}_{2} \\ {\bf f}^{(4)}_{3} \end{array} \right ] = {Q^{(3)^T}}\tilde{{\bf f}}^{(3)}= {Q^{(3)^T}} \left [ \begin{array}{c} {{\bf f}}^{(3)}_{1} \\ {{\bf f}}^{(3)}_3 \\ {\boldsymbol{0}} \end{array} \right ]. $$

Observe that \({\bf y}^{(4)}_{3}= \tilde{{\bf x}}_{1} \) and \({\bf f}^{(4)}_{3} = {\boldsymbol{0}}\), because of the structure of Q (3). Since \(\operatorname{Inertia}(M^{(4)})=\operatorname{Inertia}(\tilde {M}^{(3)})=(q+n-s,0,n-s)\), then, by [6], the submatrix X (4) of M (4) is symmetric negative definite with Cholesky factorization \(X^{(4)} = -L^{(4)} L^{(4)^{T}}\), \(L^{(4)} \in{\mathbb{R}}^{q \times q}\) nonsingular lower triangular.

We can now solve the linear system (3.17) in the following steps.

  • Observe that \({\bf y}_{1}^{(4)} = {\boldsymbol{0}}\), since \({\bf f}^{(4)}_{3}= {\boldsymbol{0}}\). Therefore the \({\bf y}_{1}^{(4)}={\boldsymbol{0}}\) is the solution of the upper anti-triangular linear system

    $$Y^{(4)^T} {\bf y}_1^{(4)}={\bf f}_3^{(4)}; $$
  • update the right-hand-side

    $$\left [ \begin{array}{c} \tilde{{\bf f}}^{(4)}_1 \\ \tilde{{\bf f}}^{(4)}_2 \end{array} \right ]= \left [ \begin{array}{c} {{\bf f}}^{(4)}_1 \\ {{\bf f}}^{(4)}_2 \end{array} \right ]- \left [ \begin{array}{c} W^{(4)} \\ {Y^{(4)}}^T \end{array} \right ] {\bf y}_1^{(4)}; $$
  • solve the linear system \(X^{(4)} {\bf y}_{2}^{(4)}= \tilde{{\bf f}}^{(4)}_{2} \),

    $$\begin{aligned} &L^{(4)} {\bf t}= -\tilde{{\bf f}}^{(4)}_2 \\ &L^{(4)^T} {\bf y}_2^{(4)}= {\bf t}; \end{aligned}$$
  • solve the upper anti-triangular linear system

    $${Y^{(4)}} {\bf y}^{(4)}_{3}={Y^{(4)}} \tilde{{\bf x}}_1=\tilde{{\bf f}}^{(4)}_1 -Z^{(4)}{\bf y}_2^{(4)}. $$

Once \(\tilde{{\bf x}}_{1} \) is computed, the solution \({\bf x} \) of the problem (1.4) can be obtained as

$${\bf x}= \tilde{Q}^{(1)} \left [ \begin{array}{c} \tilde{{\bf x}}_1 \\ \tilde{{\bf x}}_2 \end{array} \right ]. $$

If one is also interested in the computation of the solution of the augmented system (1.7), from (6.6) can be computed and, therefore, \({{\bf y}}^{(3)}\), since, by (6.3), \({{\bf s}}^{(1)}_{2}\) is already computed. Finally, from (6.2), \({\bf y}^{(2)}= Q^{(2)}{\bf y}^{(3)}\) can be computed.

About the computational complexity of this step, the computation of R 11 and R 12 requires 2(ns)2(p−(ns)/3) and 2q 2(q+ns) floating point operations, respectively. Moreover, the computation of Y (4) requires 4q(nsq)2+2q 2(ns)−2/3q 3 floating point operations.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mastronardi, N., Van Dooren, P. An algorithm for solving the indefinite least squares problem with equality constraints. Bit Numer Math 54, 201–218 (2014). https://doi.org/10.1007/s10543-013-0452-2

Download citation

Keywords

  • Indefinite matrix
  • Indefinite least squares
  • Equality constraints

Mathematics Subject Classification (2010)

  • 65F20
  • 65G05
  • 15A06