Constructive Approximation

, Volume 48, Issue 3, pp 501–521 | Cite as

Stochastic Subspace Correction in Hilbert Space

  • Michael Griebel
  • Peter Oswald


We consider an incremental approximation method for solving variational problems in infinite-dimensional separable Hilbert spaces, where in each step a randomly and independently selected subproblem from an infinite collection of subproblems is solved. We show that convergence rates for the expectation of the squared error can be guaranteed under weaker conditions than previously established in Griebel and Oswald (Constr Approx 44(1):121–139, 2016).


Infinite space splitting Subspace correction Multiplicative Schwarz Block coordinate descent Greedy Randomized Convergence rates Online learning 

Mathematics Subject Classification

65F10 65N22 49M27 



M. Griebel was partially supported by the project EXAHD of the DFG priority program 1648 Software for Exascale Computing (SPPEXA) and by the Sonderforschungsbereich 1060 The Mathematics of Emergent Effects funded by the Deutsche Forschungsgemeinschaft. This paper was written while P. Oswald held a Bonn Research Chair sponsored by the Hausdorff Center for Mathematics at the University of Bonn funded by the Deutsche Forschungsgemeinschaft. He is grateful for this support.


  1. 1.
    Barron, A., Cohen, A., Dahmen, W., DeVore, R.: Approximation and learning by greedy algorithms. Ann. Stat. 3, 64–94 (2008)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Binev, P., Cohen, A., Mula, O., Nichols, J.: Greedy algorithms for optimal measurements selection in state estimation using reduced models. SIAM/ASA J. Uncertain. Quantif. 6(3), 1101–1126 (2018)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bogachev, V.: Gaussian Measures. AMS, Providence (1998)CrossRefGoogle Scholar
  4. 4.
    Da Prato, G., Zabczyk, J.: Stochastic Equations in Infinite Dimensions. Cambridge University Press, Cambridge (1992)CrossRefGoogle Scholar
  5. 5.
    Dieuleveut, A., Bach, F.: Nonparametric stochastic approximation with large step-sizes. Ann. Stat. 44(4), 1363–1399 (2016)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Fercoq, O., Richtarik, P.: Optimization in high dimensions via accelerated, parallel, and proximal coordinate descent. SIAM Rev. 58(4), 739–771 (2016)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Griebel, M., Oswald, P.: On the abstract theory of additive and multiplicative Schwarz algorithms. Numer. Math. 70, 163–180 (1995)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Griebel, M., Oswald, P.: Greedy and randomized versions of the multiplicative Schwarz method. Linear Algebra Appl. 437(7), 1596–1610 (2012)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Griebel, M., Oswald, P.: Schwarz iterative methods: infinite space splittings. Constr. Approx. 44(1), 121–139 (2016)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Lin, J., Zhou, D.-X.: Learning theory of randomized Kaczmarz algorithm. J. Mach. Learn. Res. 16, 3341–3365 (2015)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Tarrés, P., Yao, Y.: Online learning as stochastic approximation of regularization paths: optimality and almost-sure convergence. IEEE Trans. Inf. Theory 60(9), 5716–5735 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Temlyakov, V.: Greedy Approximation. Cambridge University Press, Cambridge (2012)zbMATHGoogle Scholar
  13. 13.
    Smale, S., Zhou, D.-X.: Learning Theory: An Approximation Theory Viewpoint. Cambridge University Press, Cambridge (2007)Google Scholar
  14. 14.
    Wendland, H.: Scattered Data Approximation. Cambridge University Press, Cambridge (2010)zbMATHGoogle Scholar
  15. 15.
    Xu, J.: Iterative methods by space decomposition and subspace correction. SIAM Rev. 34(4), 581–613 (1992)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute for Numerical SimulationUniversität BonnBonnGermany
  2. 2.Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)Schloss BirlinghovenSankt AugustinGermany

Personalised recommendations