Computing

, Volume 84, Issue 1–2, pp 1–25

# Fitting multidimensional data using gradient penalties and the sparse grid combination technique

Article

## Abstract

Sparse grids, combined with gradient penalties provide an attractive tool for regularised least squares fitting. It has earlier been found that the combination technique, which builds a sparse grid function using a linear combination of approximations on partial grids, is here not as effective as it is in the case of elliptic partial differential equations. We argue that this is due to the irregular and random data distribution, as well as the proportion of the number of data to the grid resolution. These effects are investigated both in theory and experiments. As part of this investigation we also show how overfitting arises when the mesh size goes to zero. We conclude with a study of modified “optimal” combination coefficients who prevent the amplification of the sampling noise present while using the original combination coefficients.

### Keywords

Sparse grids Combination technique Regression High-dimensional data Regularisation

### Mathematics Subject Classification (2000)

62G08 65D10 62J02

## Preview

### References

1. 1.
Braess D (2001) Finite elements, 2nd edn. Cambridge University Press, Cambridge
2. 2.
Bramble JH, Xu J (1991) Some estimates for a weighted L 2 projection. Math Comput 56: 463–476
3. 3.
Brenner SC, Scott LR (2002) The mathematical theory of finite element methods. Texts in applied mathematics, 2nd edn, vol 15. Springer, New YorkGoogle Scholar
4. 4.
Bungartz H-J, Griebel M, Rüde U (1994) Extrapolation, combination, and sparse grid techniques for elliptic boundary value problems. Comput Methods Appl Mech Eng 116: 243–252
5. 5.
Garcke J (2004) Maschinelles Lernen durch Funktionsrekonstruktion mit verallgemeinerten dünnen Gittern. Doktorarbeit, Institut für Numerische Simulation, Universität BonnGoogle Scholar
6. 6.
Garcke J (2006) Regression with the optimised combination technique. In: Cohen W, Moore A (eds) Proceedings of the 23rd ICML ’06. ACM Press, New York, NY, USA, pp 321–328
7. 7.
Garcke J (2008) An optimised sparse grid combination technique for eigenproblems. In: Proceedings of ICIAM 2007, PAMM, vol 7, pp 1022301–1022302Google Scholar
8. 8.
Garcke J, Griebel M (2002) Classification with sparse grids using simplicial basis functions. Intelligent data analysis 6:483–502 (shortened version appeared in KDD 2001, Proc. of the Seventh ACM SIGKDD, F. Provost and R. Srikant (eds), pp 87–96, ACM, 2001)Google Scholar
9. 9.
Garcke J, Griebel M, Thess M (2001) Data mining with sparse grids. Computing 67: 225–253
10. 10.
Gilbarg D, Trudinger NS (2001) Elliptic partial differential equations of second order. Classics in mathematics. Springer, BerlinGoogle Scholar
11. 11.
Griebel M, Schneider M, Zenger C (1992) A combination technique for the solution of sparse grid problems. In: Groen P, Beauwens R (eds) Iterative methods in linear algebra. IMACS, Elsevier, North Holland, pp 263–281Google Scholar
12. 12.
Hackbusch W (1992) Elliptic differential equations. Springer series in computational mathematics, vol 18. Springer, BerlinGoogle Scholar
13. 13.
Hegland M (2003) Additive sparse grid fitting. In: Proceedings of the fifth international conference on curves and surfaces, Saint-Malo, France 2002, pp 209–218. Nashboro PressGoogle Scholar
14. 14.
Hegland M, Garcke J, Challis V (2007) The combination technique and some generalisations. Linear Algebra Appl 420: 249–275
15. 15.
Natterer F (1977) Regularisierung schlecht gestellter Probleme durch Projektionsverfahren. Numer Math 28: 329–341
16. 16.
Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, CambridgeGoogle Scholar
17. 17.
Tikhonov AN, Arsenin VA (1977) Solutions of ill-posed problems. W.H. Winston, Washington D.C.
18. 18.
Wahba G (1990) Spline models for observational data. Series in applied mathematics, vol 59. SIAM, PhiladelphiaGoogle Scholar