Numerical Algorithms

, Volume 80, Issue 1, pp 135–234 | Cite as

Comments on “Anderson Acceleration, Mixing and Extrapolation”

  • Donald G. M. AndersonEmail author
Original Paper


The Extrapolation Algorithm is a technique devised in 1962 for accelerating the rate of convergence of slowly converging Picard iterations for fixed point problems. Versions to this technique are now called Anderson Acceleration in the applied mathematics community and Anderson Mixing in the physics and chemistry communities, and these are related to several other methods extant in the literature. We seek here to broaden and deepen the conceptual foundations for these methods, and to clarify their relationship to certain iterative methods for root-finding problems. For this purpose, the Extrapolation Algorithm will be reviewed in some detail, and selected papers from the existing literature will be discussed, both from conceptual and implementation perspectives.


Fixed point problems Picard iteration Convergence acceleration Anderson Acceleration Anderson Mixing Root-finding problems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, D.G.: Iterative procedures for nonlinear integral equations. J. Assoc. Comput. Mach. 12, 547–560 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bierlaire, M., Crittin, F.: Solving noisy, large-scale fixed point problems and systems of nonlinear equations. Transp. Sci. 40, 44–63 (2006)CrossRefGoogle Scholar
  3. 3.
    Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)CrossRefzbMATHGoogle Scholar
  4. 4.
    Broyden, C.J.: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19, 577–593 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Calef, M.T., Fichtl, E.D., Warsa, B., Carlson, N.N.: Nonlinear Krylov acceleration applied to a discrete ordinates formulation of the k-eigenvalue problem. J. Comput. Phys. 238, 188–209 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Carlson, N.N., Miller, K.: Design and application of a gradient-weighted moving finite element code I: one dimension. SIAM J. Sci. Comput. 19, 728–765 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Eyert, V.: A comparative study on methods for convergence acceleration of iterative vector sequences. J. Comput. Phys. 124, 271–285 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Fang, H.-r., Saad, Y.: Two classes of multisecant methods for nonlinear acceleration. Numer. Linear Algebra Appl. 16, 197–221 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. The Johns Hopkins University Press, Baltimore (2013)Google Scholar
  10. 10.
    Horn, R.A., Johnson, C.A.: Matrix Analysis. Cambridge University Press, Cambridge (1985)CrossRefzbMATHGoogle Scholar
  11. 11.
    Marks, L.D., Luke, D.R.: Robust mixing for ab initio quantum mechanical calculations. Phys. Rev. B 78, 075114 (2008)CrossRefGoogle Scholar
  12. 12.
    Ni, P.: Anderson Acceleration of Fixed-Point Iteration with Applications to Electronic Structure Computations, Ph.D. thesis, Worcester Polytechnic Institute, Worcester, MA (2009)Google Scholar
  13. 13.
    Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, San Diego (1970)zbMATHGoogle Scholar
  14. 14.
    Ostrowski, A.M.: Solution of Equations and Systems of Equations, 3nd edn. Academic Press, San Diego (1966)Google Scholar
  15. 15.
    Toth, A., Kelley, C.T.: Convergence analysis for Anderson Acceleration. SIAM J. Numer. Anal. 53, 805–819 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Walker, H.F., Ni, P.: Anderson Acceleration for fixed-point iterations. SIAM J. Numer. Anal. 49, 1715–1735 (2011)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Gordon McKay Professor of Applied Mathematics, EmeritusHarvard John A. Paulson School of Engineering and Applied SciencesCambridgeUSA

Personalised recommendations