Randomized and fault-tolerant method of subspace corrections

  • Xiaozhe Hu
  • Jinchao XuEmail author
  • Ludmil T. Zikatanov


In this paper, we consider the iterative method of subspace corrections with random ordering. We prove identities for the expected convergence rate and use these results to provide sharp estimates for the expected error reduction per iteration. We also study the fault-tolerant features of the randomized successive subspace correction method by rejecting corrections when faults occur and show that the resulting iterative method converges with probability one. In addition, we derive estimates on the expected convergence rate for the fault-tolerant, randomized, subspace correction method.


Method of subspace corrections Randomized method Fault-tolerant method 


Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.


  1. 1.
    Bramble, J.: Multigrid Methods. Chapman & Hall/CRC, Boca Raton (1993)zbMATHGoogle Scholar
  2. 2.
    Cui, T., Xu, J., Zhang, C.S.: An error-resilient redundant subspace correction method. Comput. Vis. Sci. 18(2–3), 65–77 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Durstenfeld, R.: Algorithm 235: random permutation. Commun. ACM 7(7), 420 (1964). CrossRefGoogle Scholar
  4. 4.
    Eldar, Y.C., Needell, D.: Acceleration of randomized Kaczmarz method via the Johnson–Lindenstrauss Lemma. Numer. Algorithms 58(2), 163–177 (2011)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Fisher, R.A., Yates, F.: Statistical Tables for Biological Agricultural and Medical Research. Oliver and Boyd, London (1948)zbMATHGoogle Scholar
  6. 6.
    Griebel, M., Oswald, P.: On the abstract theory of additive and multiplicative Schwarz algorithms. Numer. Math. 70(2), 163–180 (1995). MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Griebel, M., Oswald, P.: Greedy and randomized versions of the multiplicative Schwarz method. Linear Algebra Appl. 437(7), 1596–1610 (2012)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Hackbusch, W.: Multigrid Methods and Applications, Springer Series in Computational Mathematics, vol. 4. Springer-Verlag, Berlin (1985)Google Scholar
  9. 9.
    Hoemmen, M., Heroux, M.A.: Fault-tolerant iterative methods via selective reliability. In: Proceedings of the 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC). IEEE Computer Society, vol. 3, p. 9 (2011)Google Scholar
  10. 10.
    Huber, M., Gmeiner, B., Rüde, U., Wohlmuth, B.: Resilience for massively parallel multigrid solvers. SIAM J. Sci. Comput. 38(5), S217–S239 (2016)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Leventhal, D., Lewis, A.S.: Randomized methods for linear constraints: convergence rates and conditioning. Math. Oper. Res. 35(3), 641–654 (2010)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Liu, J., Wright, S.J.: An accelerated randomized kaczmarz algorithm. arXiv preprint arXiv:1310.2887 (2013)
  13. 13.
    Mansour, H., Yilmaz, O.: A fast randomized kaczmarz algorithm for sparse solutions of consistent linear systems. arXiv preprint arXiv:1305.3803 (2013)
  14. 14.
    Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT Numer. Math. 50(2), 395–403 (2010)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22(2), 341–362 (2012)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Oswald, P., Zhou, W.: Convergence analysis for Kaczmarz-type methods in a Hilbert space framework. Linear Algebra Appl. 478, 131–161 (2015)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Quarteroni, A., Valli, A.: Domain Decomposition Methods for Partial Differential Equations. Oxford University Press, Oxford (1999)zbMATHGoogle Scholar
  18. 18.
    Richtárik, P., Takáč, M.: Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Math. Program. 144(1–2), 1–38 (2014)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Roy-Chowdhury, A., Banerjee, P.: A fault-tolerant parallel algorithm for iterative solution of the laplace equation. In: International Conference on Parallel Processing, 1993. ICPP 1993, vol. 3, pp. 133–140. IEEE (1993)Google Scholar
  20. 20.
    Roy-Chowdhury, A., Bellas, N., Banerjee, P.: Algorithm-based error-detection schemes for iterative solution of partial differential equations. IEEE Trans. Comput. 45(4), 394–407 (1996)CrossRefGoogle Scholar
  21. 21.
    Shantharam, M., Srinivasmurthy, S., Raghavan, P.: Fault tolerant preconditioned conjugate gradient for sparse linear system solution. In: Proceedings of the 26th ACM International Conference on Supercomputing, pp. 69–78. ACM (2012)Google Scholar
  22. 22.
    Southwell, R.V.: Relaxation Methods in Engineering Science—A Treatise in Approximate Computation. Oxford University Press, Oxford (1946)Google Scholar
  23. 23.
    Stoyanov, M.K., Webster, C.G.: Numerical analysis of fixed point algorithms in the presence of hardware faults. Tech. rep., Tech. rep. Oak Ridge National Laboratory (ORNL) (2013)Google Scholar
  24. 24.
    Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15(2), 262–278 (2009)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Toselli, A., Widlund, O.: Domain Decomposition Methods-Algorithms and Theory. Springer Verlag, Berlin (2005)CrossRefGoogle Scholar
  26. 26.
    Trottenberg, U., Oosterlee, C., Schüller, A.: Multigrid. Academic Press, Cambridge (2001)zbMATHGoogle Scholar
  27. 27.
    Xu, J.: Iterative methods by space decomposition and subspace correction. SIAM Rev. 34(4), 581–613 (1992). MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Xu, J., Zikatanov, L.: The method of alternating projections and the method of subspace corrections in Hilbert space. J. Am. Math. Soc. 15(3), 573–597 (2002). MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of MathematicsTufts UniversityMedfordUSA
  2. 2.Department of MathematicsThe Pennsylvania State UniversityUniversity ParkUSA

Personalised recommendations