Skip to main content
SpringerLink
Log in

Geometrical inverse matrix approximation for least-squares problems and acceleration strategies

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

We extend the geometrical inverse approximation approach to the linear least-squares scenario. For that, we focus on the minimization of \(1-\cos \limits (X(A^{T}A),I)\), where A is a full-rank matrix of size m × n, with m ≥ n, and X is an approximation of the inverse of ATA. In particular, we adapt the recently published simplified gradient-type iterative scheme MinCos to the least-squares problem. In addition, we combine the generated convergent sequence of matrices with well-known acceleration strategies based on recently developed matrix extrapolation methods, and also with some line search acceleration schemes which are based on selecting an appropriate steplength at each iteration. A set of numerical experiments, including large-scale problems, are presented to illustrate the performance of the different accelerations strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The Matlab package EPSfun is freely available at http://www.netlib.org/numeralgo/

References

  1. Anderson, D.G.: Iterative procedures for nonlinear integral equations. J. ACM 12, 547–560 (1965)

    Article  MathSciNet  Google Scholar 

  2. Andreani, R., Raydan, M., Tarazaga, P.: On the geometrical structure of symmetric matrices. Linear Algebra Appl. 438, 1201–1214 (2013)

    Article  MathSciNet  Google Scholar 

  3. Brezinski, C.: Convergence acceleration during the 20th century. J. Comput. Appl. Math. 122, 1–21 (2000). Numerical Analysis in the 20th Century Vol. II: Interpolation and Extrapolation

    Article  MathSciNet  Google Scholar 

  4. Brezinski, C., Redivo-Zaglia, M.: Extrapolation methods theory and practice. North-Holland, Amsterdam (1991)

    MATH  Google Scholar 

  5. Brezinski, C., Redivo-Zaglia, M.: The simplified topological ε-algorithms for accelerating sequences in a vector space. SIAM J. Sci. Comput. 36, 2227–2247 (2014)

    Article  MathSciNet  Google Scholar 

  6. Brezinski, C., Redivo-Zaglia, M.: The simplified topological ε-algorithms: software and applications. Numer. Algor. 74, 1237–1260 (2017)

    Article  MathSciNet  Google Scholar 

  7. Brezinski, C., Redivo-Zaglia, M.: The genesis and early developments of Aitkens process, Shanks transformation, the ε-algorithm, and related fixed point methods. Numer. Algor. 84, 11–133 (2019)

    Article  MathSciNet  Google Scholar 

  8. Brezinski, C., Redivo-Zaglia, M., Saad, Y.: Shanks sequence transformations and Anderson acceleration. SIAM Rev. 60, 646–669 (2018)

    Article  MathSciNet  Google Scholar 

  9. Carr, L.E., Borges, C.F., Giraldo, F.X.: An element based spectrally optimized approximate inverse preconditioner for the Euler equations. SIAM J. Sci. Comput. 34, 392–420 (2012)

    Article  MathSciNet  Google Scholar 

  10. Chehab, J.-P.: Matrix differential equations and inverse preconditioners. Comput. Appl. Math. 26, 95–128 (2007)

    MathSciNet  MATH  Google Scholar 

  11. Chehab, J.-P.: Sparse approximations of matrix functions via numerical integration of ODEs. Bull. Comput. Appl. Math. 4, 95–132 (2016)

    MathSciNet  MATH  Google Scholar 

  12. Chehab, J.P., Raydan, M.: Geometrical properties of the Frobenius condition number for positive definite matrices. Linear Algebra Appl. 429, 2089–2097 (2008)

    Article  MathSciNet  Google Scholar 

  13. Chehab, J.P., Raydan, M.: Geometrical inverse preconditioning for symmetric positive definite matrices. Mathematics 4(3), 46 (2016). https://doi.org/10.3390/math4030046

    Article  Google Scholar 

  14. Chen, K.: An analysis of sparse approximate inverse preconditioners for boundary integral equations. SIAM J. Matrix Anal. Appl. 22, 1058—1078 (2001)

    MathSciNet  MATH  Google Scholar 

  15. Chow, E., Saad, Y.: Approximate inverse techniques for block-partitioned matrices. SIAM J. Sci. Comput. 18, 1657–1675 (1997)

    Article  MathSciNet  Google Scholar 

  16. Chung, J., Chung, M., O’Leary, D.P.: Optimal regularized low rank inverse approximation. Linear Algebra Appl. 468, 260–269 (2015)

    Article  MathSciNet  Google Scholar 

  17. Cui, X., Hayami, K.: Generalized approximate inverse preconditioners for least squares problems. J.pan J. Indust. Appl. Math. 26, 1–14 (2009)

    Article  MathSciNet  Google Scholar 

  18. De Asmundis, R., di Serafino, D., Riccio, F., Toraldo, G.: On spectral properties of steepest descent methods. IMA J. Numer. Anal. 33, 1416–1435 (2013)

    Article  MathSciNet  Google Scholar 

  19. Delahaye, J.P., Germain-Bonne, B.: Résultats négatifs en accélération de la convergence. Numer. Math. 35, 443–457 (1980)

    Article  MathSciNet  Google Scholar 

  20. Diniz-Ehrhardt, M.A., Martínez, J.M., Raydan, M.: A derivative-free nonmonotone line search technique for unconstrained optimization. J. Comput. Appl. Math. 219, 383–397 (2008)

    Article  MathSciNet  Google Scholar 

  21. Forsman, K., Gropp, W., Kettunen, L., Levine, D., Salonen, J.: Solution of dense systems of linear equations arising from integral equation formulations. Antennas and Propagation Magazine 37, 96–100 (1995)

    Article  Google Scholar 

  22. Frassoldati, G., Zanni, L., Zanghirati, G.: New adaptive stepsize selections in gradient methods. J. Ind. Manag. Optim. 4, 299–312 (2008)

    MathSciNet  MATH  Google Scholar 

  23. Glunt, W., Hayden, T.L.: M. Raydan molecular conformations from distance matrices. J. Comput. Chem. 14, 114–120 (1993)

    Article  Google Scholar 

  24. Gould, N.I.M., Scott, J.A.: Sparse approximate-inverse preconditioners using norm-minimization techniques. SIAM J. Sci. Comput. 19, 605–625 (1998)

    Article  MathSciNet  Google Scholar 

  25. Graves-Morris, P.R., Roberts, P.R., Salam, A.: The epsilon algorithm and related topics. J. Comput. Appl. Math. 122, 51–80 (2000)

    Article  MathSciNet  Google Scholar 

  26. Helsing, J.: Approximate inverse preconditioners for some large dense random electrostatic interaction matrices. BIT Numer. Math. 46, 307–323 (2006)

    Article  MathSciNet  Google Scholar 

  27. Henderson, N.C., Varadhan, R.: Damped Anderson acceleration with restarts and monotonicity control for accelerating EM and EM like algorithms, Journal of Computational and Graphical Statistics. https://doi.org/10.1080/10618600.2019.1594835 (2019)

  28. Higham, N., Strabić, N.: Anderson acceleration of the alternating projections method for computing the nearest correlation matrix. Numer. Algor. 72, 1021–1042 (2016)

    Article  MathSciNet  Google Scholar 

  29. Iannazzo, B., Porcelli, M.: The Riemannian Barzilai-Borwein method with nonmonotone line-search and the matrix geometric mean computation. IMA J. Numer. Anal. 38, 495–517 (2018)

    Article  MathSciNet  Google Scholar 

  30. Jbilou, K., Messaoudi, A.: Block extrapolation methods with applications. Appl. Numer. Math. 106, 154–164 (2016)

    Article  MathSciNet  Google Scholar 

  31. Jbilou, K., Sadok, H.: Vector extrapolation methods: applications and numerical comparison. J. Comput. Appl. Math. 122, 149–165 (2000)

    Article  MathSciNet  Google Scholar 

  32. Jbilou, K., Sadok, H.: Matrix polynomial and epsilon-type extrapolation methods with applications. Numer. Algor. 68, 107–119 (2015)

    Article  MathSciNet  Google Scholar 

  33. Matos, A.C.: Convergence and acceleration properties for the vector 𝜖-algorithm. Numer. Algor. 3, 313–320 (1992)

    Article  MathSciNet  Google Scholar 

  34. Matrix Market, http://math.nist.gov/MatrixMarket/

  35. Montero, G., González, L., Flórez, E., García, M.D., Suárez, A.: Approximate inverse computation using Frobenius inner product. Numerical Linear Algebra with Applications 9, 239–247 (2002)

    Article  MathSciNet  Google Scholar 

  36. Raydan, M., Svaiter, B.: Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Comput. Optim. Appl. 21, 155–167 (2002)

    Article  MathSciNet  Google Scholar 

  37. Sajo-Castelli, A.M., Fortes, M.A., Raydan, M.: Preconditioned conjugate gradient method for finding minimal energy surfaces on Powell-Sabin triangulations. J. Comput. Appl. Math. 268, 34–55 (2014)

    Article  MathSciNet  Google Scholar 

  38. di Serafino, D., Ruggiero, V., Toraldo, G., Zanni, L.: On the steplength selection in gradient methods for unconstrained optimization. Appl. Math. Comput. 318, 176–195 (2018)

    MathSciNet  MATH  Google Scholar 

  39. Sidje, R.B., Saad, Y.: Rational approximation to the Fermi–Dirac function with applications in density functional theory. Numer. Algor. 56, 455–479 (2011)

    Article  MathSciNet  Google Scholar 

  40. Walker, H.F., Ni, P.: Anderson acceleration for fixed-point iterations. SIAM J. Numer. Anal. 49, 1715–1735 (2011)

    Article  MathSciNet  Google Scholar 

  41. Wang, J.-K., Lin, S.D.: Robust inverse covariance estimation under noisy measurements. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14), 928–936 (2014)

  42. Zhou, B., Gao, L., Dai, Y.H.: Gradient methods with adaptive step-sizes. Comput. Optim. Appl. 35, 69–86 (2006)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

We would like to thank two anonymous referees for their constructive comments and suggestions that helped us improve the final version of this paper.

Funding

This study is financially supported by CNRS throughout a 3-month Poste Rouge stay, in LAMFA Laboratory, (UMR CNRS 7352) at Université de Picardie Jules Verne, Amiens, France, from February 1 to April 30, 2019; and also by the Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) through the project UID/MAT/00297/2019 (CMA), starting on May 1, 2019.

Author information

Authors and Affiliations

  1. LAMFA, UMR CNRS, 7352, Université de Picardie Jules Verne, 33 rue Saint Leu, 80039, Amiens, France

    Jean-Paul Chehab

  2. Centro de Matemática e Aplicações (CMA), FCT, UNL, 2829-516, Caparica, Portugal

    Marcos Raydan

Authors
  1. Jean-Paul Chehab
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcos Raydan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcos Raydan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chehab, JP., Raydan, M. Geometrical inverse matrix approximation for least-squares problems and acceleration strategies. Numer Algor 85, 1213–1231 (2020). https://doi.org/10.1007/s11075-019-00862-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-019-00862-z

Keywords

Search

Navigation