Numerische Mathematik

, Volume 139, Issue 1, pp 247–280 | Cite as

Sparse approximation of multilinear problems with applications to kernel-based methods in UQ

  • Fabio Nobile
  • Raúl Tempone
  • Sören Wolfers


We provide a framework for the sparse approximation of multilinear problems and show that several problems in uncertainty quantification fit within this framework. In these problems, the value of a multilinear map has to be approximated using approximations of different accuracy and computational work of the arguments of this map. We propose and analyze a generalized version of Smolyak’s algorithm, which provides sparse approximation formulas with convergence rates that mitigate the curse of dimension that appears in multilinear approximation problems with a large number of arguments. We apply the general framework to response surface approximation and optimization under uncertainty for parametric partial differential equations using kernel-based approximation. The theoretical results are supplemented by numerical experiments.

Mathematics Subject Classification

41A05 41A25 41A63 65B99 65C05 65D10 65D32 



S. Wolfers and R. Tempone are members of the KAUST Strategic Research Initiative, Center for Uncertainty Quantification in Computational Sciences and Engineering. R. Tempone received support from the KAUST CRG3 Award Ref: 2281. F. Nobile received support from the Center for ADvanced MOdeling Science (CADMOS). We thank Abdul-Lateef Haji-Ali for many helpful discussions.


  1. 1.
    Alexanderian, A., Petra, N., Stadler, G., Ghattas, O.: Mean-variance risk-averse optimal control of systems governed by PDEs with random parameter fields using quadratic approximations. arXiv:1602.07592 (2016)
  2. 2.
    Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Chkifa, A., Cohen, A., Schwab, C.: Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs. J. Math. Pures Appl. 103(2), 400–428 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Dereich, S., Mueller-Gronbach, T.: General multilevel adaptations for stochastic approximation algorithms. arXiv:1506.0548 (2015)
  5. 5.
    Dong, Z., Georgoulis, E.H., Levesley, J., Usta, F.: Fast multilevel sparse Gaussian kernels for high-dimensional approximation and integration. arXiv:1501.03296 (2015)
  6. 6.
    Dung, D.: Continuous algorithms in n-term approximation and non-linear widths. J. Approx. Theory 102(2), 217–242 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Fasshauer, G., McCourt, M.: Kernel-Based Approximation Methods Using MATLAB. World Scientific, Singapore (2016)zbMATHGoogle Scholar
  8. 8.
    Georgoulis, E.H., Levesley, J., Subhan, F.: Multilevel sparse kernel-based interpolation. SIAM J. Sci. Comput. 35(2), A815–A831 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18(3–4), 209–232 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Gerstner, T., Heinz, S.: Dimension-and time-adaptive multilevel Monte Carlo methods. In: Sparse Grids and Applications, pp. 107–120. Springer (2012)Google Scholar
  11. 11.
    Giles, M.B.: Multilevel Monte Carlo path simulation. Oper. Res. 56(3), 607–617 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Griebel, M., Oettershagen, J.: On tensor product approximation of analytic functions. J. Approx. Theory 207, 348–379 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Griebel, M., Harbrecht, H.: A note on the construction of L-fold sparse tensor product spaces. Constr. Approx. 38(2), 235–251 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Griebel, M., Harbrecht, H.: On the construction of sparse tensor product spaces. Math. Comput. 82(282), 975–994 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Griebel, M., Schneider, M., Zenger, C.: A combination technique for the solution of sparse grid problems. In: de Groen, P., Beauwens, R. (eds.) Iterative Methods in Linear Algebra, pp. 263–281. Elsevier, Amsterdam (1992)Google Scholar
  16. 16.
    Hackbusch, W.: Tensor Spaces and Numerical Tensor Calculus. Springer, New York (2012)CrossRefzbMATHGoogle Scholar
  17. 17.
    Haji-Ali, A.-L., Nobile, F., Tamellini, L., Tempone, R.: Multi-index Stochastic collocation convergence rates for random PDEs with parametric regularity. Found. Comput. Math. 16(6), 1555–1605 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Haji-Ali, A.-L., Nobile, F., Tamellini, L., Tempone, R.: Multi-index stochastic collocation for random PDEs. Comput. Method. Appl. Mech. Eng. 306, 95–122 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Harbrecht, H., Peters, M., Siebenmorgen, M.: Multilevel accelerated quadrature for PDEs with log-normally distributed diffusion coefficient. SIAM/ASA J. Uncertain. Quantif. 4(1), 520–551 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Harbrecht, H., Peters, M., Siebenmorgen M.: On multilevel quadrature for elliptic stochastic partial differential equations. In: Sparse Grids and Applications, pp. 161–179. Springer, New York (2012)Google Scholar
  21. 21.
    Heinrich, S.: Multilevel Monte Carlo methods. In: International Conference on Large-Scale Scientific Computing, pp. 58–67. Springer, New York (2001)Google Scholar
  22. 22.
    Kuo, F.Y., Schwab, C., Sloan, I.H.: Quasi-Monte Carlo finite element methods for a class of elliptic partial differential equations with random coefficients. SIAM J. Numer. Anal. 50(6), 3351–3374 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Kuo, F.Y., Scheichl, R., Schwab, C., Sloan, I.H., Ullmann, E.: Multilevel Quasi-Monte Carlo methods for lognormal diffusion problems. Math. Comput. 86(308), 2827–2860 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Nirenberg, L.: An extended interpolation inequality. Ann. Scuola Norm. Sci. 20(4), 733–737 (1966)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Novak, E., Ritter, K.: High dimensional integration of smooth functions over cubes. Numer. Math. 75(1), 79–97 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)Google Scholar
  27. 27.
    Sahinidis, N.V.: Optimization under uncertainty: state-of-the-art and opportunities. Comput. Chem. Eng. 28(6), 971–983 (2004)CrossRefGoogle Scholar
  28. 28.
    Schaback, R., Wendland, H.: Kernel techniques: from machine learning to meshless methods. Acta Numer. 15(5), 543–639 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Schreiber, A.: Die Methode von Smolyak bei der multivariaten Interpolation’. PhD thesis. Universität Göttingen (2000)Google Scholar
  30. 30.
    Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT press, Cambridge (2001)Google Scholar
  31. 31.
    Shapiro, A.: Stochastic programming approach to optimization under uncertainty. Math. Program. 112(1), 183–220 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia (2014)zbMATHGoogle Scholar
  33. 33.
    Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. Soviet Math. Dokl. 4, 240–243 (1963)zbMATHGoogle Scholar
  34. 34.
    Stein, M.L.: Interpolation of spatial data: some theory for kriging. Springer, New York (2012)Google Scholar
  35. 35.
    Teckentrup, A.L., Jantsch, P., Webster, C.G., Gunzburger, M.: A multilevel stochastic collocation method for partial differential equations with random input data. SIAM/ASA J. Uncertain. Quantif. 3(1), 1046–1074 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Wahba, G.: Interpolating Surfaces: High Order Convergence Rates and Their Associated Designs, with Application to X-ray Image Reconstruction. Technical report, DTIC (1978)Google Scholar
  37. 37.
    Wasilkowski, G.W., Wozniakowski, H.: Explicit cost bounds of algorithms for multivariate tensor product problems. J. Complex. 11(1), 1–56 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Wendland, H.: Scattered Data Approximation. Cambridge University Press, Cambridge (2004)CrossRefzbMATHGoogle Scholar
  39. 39.
    Wendland, H., Rieger, C.: Approximate interpolation with applications to selecting smoothing parameters. Numer. Math. 101(4), 729–748 (2005)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.MATHICSE-CSQIÉcole Polytechnique Fédérale de Lausanne (EPFL)LausanneSwitzerland
  2. 2.Applied Mathematics and Computational ScienceKing Abdullah University of Science and Technology (KAUST)ThuwalSaudi Arabia

Personalised recommendations