, Volume 84, Issue 1–2, pp 1–25 | Cite as

Fitting multidimensional data using gradient penalties and the sparse grid combination technique

  • Jochen Garcke
  • Markus Hegland


Sparse grids, combined with gradient penalties provide an attractive tool for regularised least squares fitting. It has earlier been found that the combination technique, which builds a sparse grid function using a linear combination of approximations on partial grids, is here not as effective as it is in the case of elliptic partial differential equations. We argue that this is due to the irregular and random data distribution, as well as the proportion of the number of data to the grid resolution. These effects are investigated both in theory and experiments. As part of this investigation we also show how overfitting arises when the mesh size goes to zero. We conclude with a study of modified “optimal” combination coefficients who prevent the amplification of the sampling noise present while using the original combination coefficients.


Sparse grids Combination technique Regression High-dimensional data Regularisation 

Mathematics Subject Classification (2000)

62G08 65D10 62J02 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Braess D (2001) Finite elements, 2nd edn. Cambridge University Press, CambridgezbMATHGoogle Scholar
  2. 2.
    Bramble JH, Xu J (1991) Some estimates for a weighted L 2 projection. Math Comput 56: 463–476zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Brenner SC, Scott LR (2002) The mathematical theory of finite element methods. Texts in applied mathematics, 2nd edn, vol 15. Springer, New YorkGoogle Scholar
  4. 4.
    Bungartz H-J, Griebel M, Rüde U (1994) Extrapolation, combination, and sparse grid techniques for elliptic boundary value problems. Comput Methods Appl Mech Eng 116: 243–252zbMATHCrossRefGoogle Scholar
  5. 5.
    Garcke J (2004) Maschinelles Lernen durch Funktionsrekonstruktion mit verallgemeinerten dünnen Gittern. Doktorarbeit, Institut für Numerische Simulation, Universität BonnGoogle Scholar
  6. 6.
    Garcke J (2006) Regression with the optimised combination technique. In: Cohen W, Moore A (eds) Proceedings of the 23rd ICML ’06. ACM Press, New York, NY, USA, pp 321–328CrossRefGoogle Scholar
  7. 7.
    Garcke J (2008) An optimised sparse grid combination technique for eigenproblems. In: Proceedings of ICIAM 2007, PAMM, vol 7, pp 1022301–1022302Google Scholar
  8. 8.
    Garcke J, Griebel M (2002) Classification with sparse grids using simplicial basis functions. Intelligent data analysis 6:483–502 (shortened version appeared in KDD 2001, Proc. of the Seventh ACM SIGKDD, F. Provost and R. Srikant (eds), pp 87–96, ACM, 2001)Google Scholar
  9. 9.
    Garcke J, Griebel M, Thess M (2001) Data mining with sparse grids. Computing 67: 225–253zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Gilbarg D, Trudinger NS (2001) Elliptic partial differential equations of second order. Classics in mathematics. Springer, BerlinGoogle Scholar
  11. 11.
    Griebel M, Schneider M, Zenger C (1992) A combination technique for the solution of sparse grid problems. In: Groen P, Beauwens R (eds) Iterative methods in linear algebra. IMACS, Elsevier, North Holland, pp 263–281Google Scholar
  12. 12.
    Hackbusch W (1992) Elliptic differential equations. Springer series in computational mathematics, vol 18. Springer, BerlinGoogle Scholar
  13. 13.
    Hegland M (2003) Additive sparse grid fitting. In: Proceedings of the fifth international conference on curves and surfaces, Saint-Malo, France 2002, pp 209–218. Nashboro PressGoogle Scholar
  14. 14.
    Hegland M, Garcke J, Challis V (2007) The combination technique and some generalisations. Linear Algebra Appl 420: 249–275zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Natterer F (1977) Regularisierung schlecht gestellter Probleme durch Projektionsverfahren. Numer Math 28: 329–341zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, CambridgeGoogle Scholar
  17. 17.
    Tikhonov AN, Arsenin VA (1977) Solutions of ill-posed problems. W.H. Winston, Washington D.C.zbMATHGoogle Scholar
  18. 18.
    Wahba G (1990) Spline models for observational data. Series in applied mathematics, vol 59. SIAM, PhiladelphiaGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.Institut für MathematikTechnische Universität BerlinBerlinGermany
  2. 2.Mathematical Sciences InstituteAustralian National UniversityCanberraAustralia

Personalised recommendations