On Linear Versus Nonlinear Approximation in the Average Case Setting

  • Leszek PlaskotaEmail author


We compare the average errors of n-term linear and nonlinear approximations assuming that the coefficients in an orthogonal expansion of the approximated element are scaled i.i.d. random variables. We show that generally the n-term nonlinear approximation can be even exponentially better than the n-term linear approximation. On the other hand, if the scaling parameters decay no faster than polynomially then the average errors of nonlinear approximations do not converge to zero faster than those of linear approximations, as n → +. The main motivation and application is the approximation of Gaussian processes. In this particular case, the nonlinear approximation is, roughly, no more than n times better than its linear counterpart.


  1. 1.
    Cioica, P.A., Dahlke, S., Döhring, N., Kinzel, S., Lindner, F., Raasch, T., Ritter, K., Schilling, R.L.: Adaptive wavelet methods for the stochastic Poisson equation. BIT Numer. Math. 52, 589–614 (2012)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Cohen, A., D’Ales, J.-P.: Nonlinear approximation of random functions. SIAM J. Appl. Math. 57, 518–540 (1997)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Cohen, A., Daubechies, I., Guleryuz, O.G., Orchard, M.T.: On the importance of combining wavelet-based nonlinear approximation with coding strategies. IEEE Trans. Inf. Theory 48, 1895–1921 (2002)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Creutzig, J., Müller-Gronbach, T., Ritter, K.: Free-knot spline approximation of stochastic processes. J. Complexity 23, 867–889 (2007)MathSciNetCrossRefGoogle Scholar
  5. 5.
    DeVore, R.A.: Nonlinear approximation. Acta Numer. 8, 51–150 (1998)CrossRefGoogle Scholar
  6. 6.
    DeVore, R.A., Jawerth, B.: Optimal nonlinear approximation. Manuscripta Math. 63, 469–478 (1992)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Kon, M.A., Plaskota, L.: Information complexity of neural networks. Neural Netw. 13, 365–376 (2000)CrossRefGoogle Scholar
  9. 9.
    Kon, M.A., Plaskota, L.: Information-based nonlinear approximation: an average case setting. J. Complexity 21, 211–229 (2005)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Kuo, H.-H.: Gaussian Measures in Banach Spaces. Lecture Notes in Mathematics, vol. 463. Springer, New York (1975)Google Scholar
  11. 11.
    Plaskota, L.: Noisy Information and Computational Complexity. Cambridge University Press, Cambridge (1996)CrossRefGoogle Scholar
  12. 12.
    Rauhut, H.: Compressive sensing and structured random matrices. In: Fornasier, M. (ed.) Theoretical Foundations and Numerical Methods for Sparse Recovery, vol. 9, pp. 1–92. Walter de Gruyter, Berlin (2012)Google Scholar
  13. 13.
    Ritter, K.: Average-Case Analysis of Numerical Problems. Springer, Berlin (2000)CrossRefGoogle Scholar
  14. 14.
    Temlyakov, V.N.: Nonlinear methods of approximation. Found. Comput. Math. 3, 33–107 (2003)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Traub, J.F., Wasilkowski, G.W., Woźniakowski, H.: Information-Based Complexity. Academic Press, New York (1988)zbMATHGoogle Scholar
  16. 16.
    Vakhania, N.N., Tarieladze, V.I., Chobanyan, S.A.: Probability Distributions on Banach Spaces. Kluwer, Dordrecht (1987)CrossRefGoogle Scholar
  17. 17.
    Vybiral, J.: Average best m-term approximation. Constr. Approx. 36, 83–115 (2012)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Mathematics, Informatics, and MechanicsUniversity of WarsawWarsawPoland

Personalised recommendations