Advertisement

QR Decomposition An Annotated Bibliography

  • Marcello L.R. de CamposEmail author
  • Gilbert Strang
Chapter

Abstract

This chapter is divided into two parts. The first one goes back in time and tries to retrace the steps of great mathematicians who laid the foundations of numerical linear algebra. We describe some early methods to compute the eigenvalues and eigenvectors associated to a matrix A. The QR decomposition (orthogonalization as in Gram–Schmidt) is encountered in many of these methods as a fundamental tool for the factorization of A. The first part describes the QR algorithm, which uses the QR decomposition iteratively for solving the eigenproblem A xx. The second part of the chapter analyzes the application of the QR decomposition to adaptive filtering.

Keywords

Toeplitz Operator Systolic Array Recursive Little Square Cholesky Factor Plane Rotation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    M. Gutknecht, The pioneer days of scientific computing in Switzerland. Proceedings of the ACM Conference on History of Scientific and Numeric Computation. Princeton, NJ, USA, pp. 63–69 (1987)Google Scholar
  2. 2.
    A. S. Householder, Bibliography on numerical analysis. Journal of the Association for Computing Machinery (JACM), vol. 3, no. 2, pp. 85–100 (April 1956)Google Scholar
  3. 3.
    G. Strang, Computational Science and Engineering. Wellesley-Cambridge Press, Wellesley, MA, USA (2007)Google Scholar
  4. 4.
    H. A. Van der Vorst and G. H. Golub, 150 years old and still alive: Eigenproblems. The State of the Art in Numerical Analysis. I. S. Duff and G. A. Watson, Eds., vol. 63 of Institute of Mathematics and Its Applications Conference Series New Series, pp. 93–120, Oxford University Press (1997)Google Scholar
  5. 5.
    G. H. Golub and H. A. van der Vorst, Eigenvalue computation in the 20th century. Journal of Computational and Applied Mathematics, vol. 123, no. 1–2, pp. 35–65 (November 2000)Google Scholar
  6. 6.
    I. Fredholm, Sur une classe d’équations fonctionnelles. Acta Mathematica, vol. 27, no. 1, pp. 365–390 (December 1903)Google Scholar
  7. 7.
    C. Lanczos, An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. Journal of Research of the National Bureau of Standards, vol. 45, no. 4, pp. 255–282 (October 1950)Google Scholar
  8. 8.
    J. Von Neumann and H. H. Goldstine, Numerical inverting of matrices of high order. Bulletin of the American Mathematical Society, vol. 53, no. 11, pp. 1021–1099 (1947)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    A. S. Householder, A survey of some closed methods for inverting matrices. Journal of the Society for Industrial and Applied Mathematics, vol. 5, no. 3, pp. 155–169 (September 1957)Google Scholar
  10. 10.
    A. S. Householder, A class of methods for inverting matrices. Journal of the Society for Industrial and Applied Mathematics, vol. 6, no. 2, pp. 189–195 (June 1958)Google Scholar
  11. 11.
    M. R. Hestenes, Inversion of matrices by biorthogonalization and related results. Journal of the Society for Industrial and Applied Mathematics, vol. 6, no. 1, pp. 51–90 (March 1958)Google Scholar
  12. 12.
    J. H. Wilkinson, The Algebraic Eigenvalue Problem. Oxford University Press, New York, NY, USA (1965)zbMATHGoogle Scholar
  13. 13.
    R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge University Press, Cambridge, UK (1985)zbMATHGoogle Scholar
  14. 14.
    G. H. Golub and C. F. Van Loan, Matrix Computations. 3rd edition The Johns Hopkins University Press, Baltimore, MD, USA (1996)zbMATHGoogle Scholar
  15. 15.
    L. N. Trefethen and D. Bau III, Numerical Linear Algebra. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, USA (1997)zbMATHGoogle Scholar
  16. 16.
    G. Strang, Linear Algebra and Its Applications. 4th edition Thomson–Brooks/Cole, Belmont, CA, USA (2005)Google Scholar
  17. 17.
    C. G. J. Jacobi, Über ein leichtes verfahren, die in der theorie der säcularstörungen vorkommenden gleichungen numerisch aufzulösen. Journal für die Reine und Angewandte Mathematik, vol. 30, pp. 51–94 (1846)zbMATHCrossRefGoogle Scholar
  18. 18.
    J. Greenstadt, A method for finding roots of arbitrary matrices. Mathematical Tables and Other Aids to Computation, vol. 9, no. 50, pp. 47–52 (April 1955)Google Scholar
  19. 19.
    H. H. Goldstine, F. J. Murray, and J. von Neumann, The Jacobi method for real symmetric matrices. Journal of the ACM, vol. 6, no. 1, pp. 59–96 (January 1959)Google Scholar
  20. 20.
    H. H. Goldstine, A History of Numerical Analysis from the 16th through the 19th Century. Studies in the History of Mathematics and Physical Sciences. Springer-Verlag, New York, NY, USA (1977)Google Scholar
  21. 21.
    G. E. Forsythe and P. Henrici, The cyclic Jacobi method for computing the principal values of a complex matrix. Transactions of the American Mathematical Society, vol. 94, no. 1, pp. 1–23 (January 1960)Google Scholar
  22. 22.
    A. Cayley, A memoir on the theory of matrices. Philosophical Transactions of the Royal Society of London, vol. CXLVIII, pp. 17–37 (1858)CrossRefGoogle Scholar
  23. 23.
    I. G. Bashmakova and A. N. Rudakov, The evolution of algebra 1800–1870. The American Mathematical Monthly, vol. 102, no. 3, pp. 266–270 (March 1995)Google Scholar
  24. 24.
    N. J. Higham, Cayley, Sylvester, and early matrix theory. Linear Algebra and Its Applications, vol. 428, no. 1, pp. 39–43 (January 2008)Google Scholar
  25. 25.
    P. Horst, A method for determining the coefficients of a characteristic equation. The Annals of Mathematical Statistics, vol. 6, no. 2, pp. 83–84 (June 1935)Google Scholar
  26. 26.
    W. Givens, The characteristic value-vector problem. Journal of the ACM, vol. 4, no. 3, pp. 298–307 (July 1957)Google Scholar
  27. 27.
    N. H. Abel. Démonstration de l’impossibilité de la résolution algébrique des équations générales qui passent le quatrième degré. Oeuvres Complètes. Suivi de Niels Henrik Abel, Sa Vie et Son Action Scientifique, deuxiéme edition Jacques Gabay, vol. I, pp. 66–87 (1992)Google Scholar
  28. 28.
    N. H. Abel. Mémoire sur les équations algébriques, où lx02019; on démontre l’impossibilité de la résolution de l’équation générale du cinquième degré. Oeuvres Complètes. Suivi de Niels Henrik Abel, Sa Vie et Son Action Scientifique, deuxiéme edition Jacques Gabay, vol. I, pp. 28–33 (1992)MathSciNetGoogle Scholar
  29. 29.
    N. H. Abel. Sur la resolution algébrique des équations. Oeuvres Complètes. Suivi de Niels Henrik Abel, Sa Vie et Son Action Scientifique, deuxiéme edition Jacques Gabay, vol. II, pp. 217–243 (1992)Google Scholar
  30. 30.
    H. Rutishauser and H. R. Schwarz, The LR transformation method for symmetric matrices. Numerische Mathematik, vol. 5, no. 1, pp. 273–289 (December 1963)Google Scholar
  31. 31.
    H. Rutishauser, Der quotienten-differenzen-algorithmus. Zeitschrift für Angewandte Mathematik und Physik, vol. 5, no. 3, pp. 233–251 (May 1954)Google Scholar
  32. 32.
    H. Rutishauser, Anwendungen des quotienten-differenzen-algorithmus. Zeitschrift für Angewandte Mathematik und Physik, vol. 5, no. 6, pp. 496–508 (November 1954)Google Scholar
  33. 33.
    H. Rutishauser, Une méthode pour la détermination des valeurs propres d’une matrice. Comptes Rendus Hebdomadaires des Séances de l’Académie des Sciences, vol. 240, pp. 34–36 (January-March 1955)Google Scholar
  34. 34.
    J. G. F. Francis, The QR transformation, a unitary analogue to the LR transformation — part 1. The Computer Journal, vol. 4, no. 3, pp. 265–271 (1961)CrossRefMathSciNetGoogle Scholar
  35. 35.
    V. N. Kublanovskaya, On some algorithms for the solution of the complete eigenvalue problem. Computational Mathematics and Mathematical Physics, vol. 3, pp. 637–657 (1961)Google Scholar
  36. 36.
    J. G. F. Francis, The QR transformation — part 2. The Computer Journal, vol. 4, no. 4, pp. 332–345 (1962)CrossRefMathSciNetGoogle Scholar
  37. 37.
    D. S. Watkins, QR-like algorithms for eigenvalue problems. Journal of Computational and Applied Mathematics, vol. 123, no. 1–2, pp. 67–83 (November 2000)Google Scholar
  38. 38.
    D. S. Watkins, The QR algorithm revisited. SIAM Review, vol. 50, no. 1, pp. 133–145 (2008)zbMATHCrossRefMathSciNetGoogle Scholar
  39. 39.
    E. Anderson, Z. Bai, C. Bischof, L. S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen. LAPACK Users’ Guide. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, USA, third edition, (1999)Google Scholar
  40. 40.
    L. S. Blackford, J. Choi, A. Cleary, E. D’Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley. ScaLAPACK Users’ Guide. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, USA (1997)zbMATHGoogle Scholar
  41. 41.
    D. Day. How the QR algorithm fails to converge and how to fix it. Technical Report 96-0913 J, Sandia National Laboratory, USA (April 1996)Google Scholar
  42. 42.
    B. N. Parlett, Convergence of the QR algorithm. Numerische Mathematik, vol. 7, no. 2, pp. 187–193 (April 1965)Google Scholar
  43. 43.
    B. N. Parlett, Global convergence of the basic QR algorithm on Hessenberg matrices. Mathematics of Computation, vol. 22, no. 104, pp. 803–817 (October 1968)Google Scholar
  44. 44.
    B. N. Parlett, The Symmetric Eigenvalue Problem. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, USA (1998)zbMATHGoogle Scholar
  45. 45.
    D. S. Watkins, Understanding the {QR} algorithm. SIAM Review, vol. 24, no. 4, pp. 427–440 (October 1982)Google Scholar
  46. 46.
    B. N. Parlett and Jr. W. G. Poole, A geometric theory for the QR, LU, and power iterations. SIAM Journal on Numerical Analysis, vol. 10, no. 2, pp. 389–412 (April 1973)Google Scholar
  47. 47.
    E. Schmidt, Zur theorie der linearen und nichtlinearen integralgleichungen i. teil: Entwicklung willkürlicher funktionen nach systemen vorgeschriebener. Mathematische Annalen, vol. 63, no. 4, pp. 433–476 (December 1907)Google Scholar
  48. 48.
    J. P. Gram, Ueber entwickelung reeller functionen in reihen mittelst der methode der kleinsten quadrate. Journal für die reine und angewandte Mathematik, vol. 94, pp. 71–73 (1883)Google Scholar
  49. 49.
    Y. K. Wong, An application of orthogonalization process to the theory of least squares. The Annals of Mathematical Statistics, vol. 6, no. 2, pp. 53–75 (June 1935)Google Scholar
  50. 50.
    S. J. Leon, W. Gander, J. Langou, and Å. Björck, Gram-Schmidt orthogonalization: 100 years and more. Manuscript to appear (2008)Google Scholar
  51. 51.
    Å. Björck, Solving linear least squares problems by Gram-Schmidt orthogonalization. BIT Numerical Mathematics, vol. 7, no. 1, pp. 1–21 (March 1967)Google Scholar
  52. 52.
    Å. Björck, Numerics of Gram-Schmidt orthogonalization. Linear Algebra and its Applications, vol. 197–198, pp. 297–316 (January–February 1994)Google Scholar
  53. 53.
    W. Gander. Algorithms for the QR-decomposition. Research Report 80-02, Eidgenoessische Technische Hochschule, Zurich, Switzerland (April 1980)Google Scholar
  54. 54.
    A. S. Householder, Unitary triangularization of a nonsymmetric matrix. Journal of the ACM, vol. 5, no. 4, pp. 339–342 (October 1958)Google Scholar
  55. 55.
    J. H. Wilkinson, Householder’s method for the solution of the algebraic eigenproblem. The Computer Journal, vol. 3, no. 1, pp. 23–27 (1960)zbMATHCrossRefMathSciNetGoogle Scholar
  56. 56.
    W. Givens, Computation of plane unitary rotations transforming a general matrix to triangular form. Journal of the Society for Industrial and Applied Mathematics, vol. 6, no. 1, pp. 26–50 (March 1958)Google Scholar
  57. 57.
    E. E. Osborne, On least squares solutions of linear equations. Journal of the ACM, vol. 8, no. 4, pp. 628–636 (October 1961)Google Scholar
  58. 58.
    A. S. Householder, The approximate solution of matrix problems. Journal of the ACM, vol. 5, no. 3, pp. 205–243 (July 1958)Google Scholar
  59. 59.
    J. H. Wilkinson, Error analysis of direct methods of matrix inversion. Journal of the ACM, vol. 8, no. 3, pp. 281–330 (July 1961)Google Scholar
  60. 60.
    Å. Björck, Iterative refinement of linear least squares solutions I. BIT Numerical Mathematics, vol. 7, no. 4, pp. 257–278 (December 1967)Google Scholar
  61. 61.
    Å. Björck, Iterative refinement of linear least squares solutions II. BIT Numerical Mathematics, vol. 8, no. 1, pp. 8–30 (March 1968)Google Scholar
  62. 62.
    Å. Björck and G. H. Golub, Iterative refinement of linear least squares solutions by Householder transformation. BIT Numerical Mathematics, vol. 7, no. 4, pp. 322–337 (December 1967)Google Scholar
  63. 63.
    Å. Björck, Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, USA (1996)zbMATHGoogle Scholar
  64. 64.
    R. W. Farebrother, Linear Least Squares Computations. Marcel Dekker, Inc., New York, NY, USA (1988)zbMATHGoogle Scholar
  65. 65.
    C. F. Gauss, Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium. Perthes and Besser, Hamburg (1809)Google Scholar
  66. 66.
    A. M. Legendre, Nouvelles Méthodes pour la Détermination des Orbites des Comètes. Courcier, Paris, France (1805)Google Scholar
  67. 67.
    G. H. Golub, Numerical methods for solving linear least squares problems. Numerische Mathematik, vol. 7, no. 3, pp. 206–216 (June 1965)Google Scholar
  68. 68.
    P. Businger and G. H. Golub, Linear least squares solutions by {H}ouseholder transformations. Numerische Mathematik, vol. 7, no. 3, pp. 269–276 (June 1965)Google Scholar
  69. 69.
    W. M. Gentleman, Least squares computations by Givens transformations without square roots. IMA Journal of Applied Mathematics, vol. 12, no. 3, pp. 329–336 (1973)zbMATHCrossRefMathSciNetGoogle Scholar
  70. 70.
    W. M. Gentleman, Basic procedures for large, sparse or weighted linear least squares problems (algorithm AS 75). Applied Statistics, vol. 23, no. 3, pp. 448–454 (1974)CrossRefGoogle Scholar
  71. 71.
    S. Hammarling, A note on the modifications to the Givens plane rotations. IMA Journal of Applied Mathematics, vol. 13, no. 2, pp. 215–218 (April 1974)Google Scholar
  72. 72.
    W. M. Gentleman, Error analysis of QR decompositions by Givens transformations. Linear Algebra and Its Applications, vol. 10, no. 3, pp. 189–197 (June 1975)Google Scholar
  73. 73.
    H. T. Kung, Why systolic architectures? Computer, vol. 15, no. 1, pp. 37–46 (January 1982)Google Scholar
  74. 74.
    W. M. Gentleman and H. T. Kung, Matrix triangularization by systolic arrays. Real-Time Signal Processing IV, SPIE Proceedings, vol. 298, pp. 19–26 (1981)Google Scholar
  75. 75.
    A. Bojanczyk, R. P. Brent, and H. T. Kung, Numerically stable solution of dense systems of linear equations using mesh-connected processors. SIAM Journal on Scientific and Statistical Computing, vol. 5, no. 1, pp. 95–104 (1984)zbMATHCrossRefMathSciNetGoogle Scholar
  76. 76.
    J. G. McWhirter, Recursive least-squares minimization using a systolic array. Real-Time Signal Processing VI, SPIE Proceedings, vol. 431, pp. 105–112 (1983)Google Scholar
  77. 77.
    J. G. McWhirter, Systolic array for recursive least-squares minimisation. Electronics Letters, vol. 19, no. 18, pp. 729–730 (September 1983)Google Scholar
  78. 78.
    C. R. Ward, P. J. Hargrave, and J. G. McWhirter, A novel algorithm and architecture for adaptive digital beamforming. IEEE Transactions on Antennas and Propagation, vol. AP-34, no. 3, pp. 338–346 (March 1986)Google Scholar
  79. 79.
    A. P. Varvitsiotis, S. Theodoridis, and G. Moustakides, A novel structure for adaptive LS FIR filtering based on QR decomposition. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’ 89, vol. 2, pp. 904–907 (May 1989)Google Scholar
  80. 80.
    J. M. Cioffi, A fast QR/frequency-domain {RLS} adaptive filter. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’87, vol. 12, pp. 407–410 (April 1987)Google Scholar
  81. 81.
    J. M. Cioffi, High-speed systolic implementation of fast QR adaptive filters. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’88, vol. 3, pp. 1584–1587 (April 1988)Google Scholar
  82. 82.
    J. M. Cioffi, The fast adaptive ROTOR’s RLS algorithm. IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 4, pp. 631–653 (April 1990)Google Scholar
  83. 83.
    M. Bellanger, A survey of QR based fast least squares adaptive filters: From principles to realization. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’91, vol. 3, pp. 1833–1836 (April 1991)Google Scholar
  84. 84.
    A. L. Ghirnikar and S. T. Alexander, Stable recursive least squares filtering using an inverse QR decomposition. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’90, vol. 3, pp. 1623–1626 (April 1990)Google Scholar
  85. 85.
    A. L. Ghirnikar, Performance and implementation of the inverse QR adaptive filter. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’92, vol. 4, pp. 29–32 (March 1992)Google Scholar
  86. 86.
    S. T. Alexander and A. L. Ghirnikar, A method for recursive least squares filtering based upon an inverse QR decomposition. IEEE Transactions on Signal Processing, vol. 41, no. 1, pp. 20–30 (January 1993)Google Scholar
  87. 87.
    J. G. McWhirter and I. K. Proudler, On the formal derivation of a systolic array for recursive least squares estimation. International Conference on Control, vol. 2, pp. 1272–1277 (March 1994)Google Scholar
  88. 88.
    M. Bellanger, The potential of QR adaptive filter variables for signal analysis. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’89, vol. 4, pp. 2166–2169 (May 1989)Google Scholar
  89. 89.
    M. Bellanger, Sur la dualité entre la prédiction linéaire d’un signal et la decomposition QR. Douzième Colloque Gretsi - Juan-les-Pins, pp. 13–16 (June 1989)Google Scholar
  90. 90.
    P. A. Regalia and M. G. Bellanger, On the duality between fast QR methods and lattice methods in least squares adaptive filtering. IEEE Transactions on Signal Processing, vol. 39, no. 4, pp. 879–891 (April 1991)Google Scholar
  91. 91.
    I. K. Proudler, T. J. Shepherd, and J. G. McWhirter, Computationally efficient QRD-based wide-band beamforming. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’90, vol. 3, pp. 1799–1802 (April 1990)Google Scholar
  92. 92.
    I. K. Proudler, J. G. McWhirter, and T. J. Shepherd, Computationally efficient QR decomposition approach to least squares adaptive filtering. IEE Proceedings F, Radar and Signal Processing, vol. 138, no. 4, pp. 341–353 (August 1991)Google Scholar
  93. 93.
    C. J. Demeure and L. L. Scharf, True lattice algorithms for square root solution of least squares linear prediction problems. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’88, vol. 4, pp. 2312–2315 (April 1988)Google Scholar
  94. 94.
    C. P. Rialan, Fast algorithms for QR and Cholesky factors of Toeplitz operators. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’87, vol. 12, pp. 41–44 (April 1987)Google Scholar
  95. 95.
    C. P. Rialan and L. L. Scharf, Fast algorithms for computing QR and Cholesky factors of Toeplitz operators. IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 36, no. 11, pp. 1740–1748 (November 1988)Google Scholar
  96. 96.
    J. G. McWhirter, Algorithmic engineering in digital signal processing. Second International Specialist Seminar on the Design and Application of Parallel Digital Processors, pp. 11–18 (April 1991)Google Scholar
  97. 97.
    J. G. McWhirter, Algorithmic engineering in adaptive signal processing. IEE Proceedings F, Radar and Signal Processing, vol. 139, no. 3, pp. 226–232 (June 1992)Google Scholar
  98. 98.
    I. K. Proudler and J. G. McWhirter, Algorithmic engineering in adaptive signal processing: worked examples. IEE Proceedings — Vision, Image and Signal Processing, vol. 141, no. 1, pp. 19–26 (February 1994)Google Scholar
  99. 99.
    I. K. Proudler, J. G. McWhirter, and M. Moonen, On the formal derivation of a systolic array for recursive least squares estimation. IEEE International Symposium on Circuits and Systems, ISCAS’94, vol. 2, pp. 329–332 (June 1994)Google Scholar
  100. 100.
    I. K. Proudler, J. G. McWhirter, M. Moonen, and G. Hekstra, Formal derivation of a systolic array for recursive least squares estimation. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 43, no. 3, pp. 247–254 (March 1996)Google Scholar
  101. 101.
    M. Moonen and I. K. Proudler, Generating ‘fast {QR}’ algorithms using signal flow graph techniques. Thirtieth Asilomar Conference on Signals, Systems and Computers, vol. 1, pp. 410–414 (November 1996)Google Scholar
  102. 102.
    M. Harteneck, J. G. McWhirter, I. K. Proudler, and R. W. Stewart, Algorithmically engineered fast multichannel adaptive filter based on QR-RLS. IEE Proceedings — Vision, Image and Signal Processing, vol. 146, no. 1, pp. 7–13 (February 1999)Google Scholar

Copyright information

© Springer-Verlag US 2009

Authors and Affiliations

  1. 1.Federal University of Rio de JaneiroRio de JaneiroBrazil
  2. 2.Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations