Classification with Probability Density Estimation on Sparse Grids

  • Benjamin PeherstorferEmail author
  • Fabian Franzelin
  • Dirk Pflüger
  • Hans-Joachim Bungartz
Conference paper
Part of the Lecture Notes in Computational Science and Engineering book series (LNCSE, volume 97)


We present a novel method to tackle the multi-class classification problem with sparse grids and show how the computational procedure can be split into an Offline phase (pre-processing) and a very rapid Online phase. For each class of the training data the underlying probability density function is estimated on a sparse grid. The class of a new data point is determined by the values of the density functions at this point. Our classification method can deal with more than two classes in a natural way and it provides a stochastically motivated confidence value which indicates how to rate the respond to a new point. Furthermore, the underlying density estimation method allows us to pre-compute the system matrix and store it in an appropriate format. This so-called Offline/Online splitting of the computational procedure allows an Online phase where only a few matrix-vector products are necessary to learn a new, previously unseen training data set. In particular, we do not have to solve a system of linear equations anymore. We show that speed ups by a factor of several hundred are possible. A typical application for such an Offline/Online splitting is cross validation. We present the algorithm and the computational procedure for our classification method, report on the employed density estimation method on sparse grids and show by means of artificial and real-world data sets that we obtain competitive results compared to the classical sparse grid classification method based on regression.


Density Estimation System Matrix Sparse Grid Online Phase Probability Density Estimation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Azzalini, N. Torelli, Clustering via nonparametric density estimation. Stat. Comput. 17(1), 71–80 (2007)CrossRefMathSciNetGoogle Scholar
  2. 2.
    B. Babcock, S. Babu, M. Datar, R. Motwani, J. Widom, Models and issues in data stream systems, in Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (ACM, New York, 2002)Google Scholar
  3. 3.
    C. Bishop, Pattern Recognition and Machine Learning (Springer, Berlin, 2007)Google Scholar
  4. 4.
    H.-J. Bungartz, M. Griebel, Sparse grids. Acta Numer. 13, 1–123 (2004)CrossRefMathSciNetGoogle Scholar
  5. 5.
    R. Fisher, The use of multiple measurements in taxonomic problems. Ann. Hum. Genet. 7(2), 179–188 (1936)Google Scholar
  6. 6.
    F. Franzelin, Classification with estimated densities on sparse grids. Master’s thesis, Institut für Informatik, Technische Universität München, 2011Google Scholar
  7. 7.
    M. Gaber, A. Zaslavsky, S. Krishnaswamy, Mining data streams: a review. SIGMOD Rec. 34(2), 18–26 (2005)CrossRefGoogle Scholar
  8. 8.
    J. Gama, Knowledge Discovery from Data Streams (Chapman & Hall, London, 2010)CrossRefzbMATHGoogle Scholar
  9. 9.
    J. Garcke, Maschinelles Lernen durch Funktionsrekonstruktion mit verallgemeinerten dünnen Gittern. Doktorarbeit, Institut für Numerische Simulation, Universität Bonn, 2004Google Scholar
  10. 10.
    J. Garcke, M. Griebel, Classification with sparse grids using simplicial basis functions. Intell. Data Anal. 6(6), 483–502 (2002)zbMATHGoogle Scholar
  11. 11.
    J. Garcke, M. Griebel, M. Thess, Data mining with sparse grids. Computing 67(3), 225–253 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning (Springer, Berlin, 2009)CrossRefzbMATHGoogle Scholar
  13. 13.
    M. Hegland, G. Hooker, S. Roberts, Finite element thin plate splines in density estimation. ANZIAM J. 42, C712–C734 (2000)MathSciNetGoogle Scholar
  14. 14.
    A. Heinecke, B. Peherstorfer, D. Pflüger, Z. Song, Sparse grid classifiers as base learners for AdaBoost, in 2012 International Conference on High Performance Computing and Simulation (HPCS), Madrid (IEEE, New York, 2012)Google Scholar
  15. 15.
    C.-W. Hsu, C.-C. Chang, C.-J. Lin, A practical guide to support vector classification. Technical report, Department of Computer Science, National Taiwan University, 2003Google Scholar
  16. 16.
    T. Jebara, Machine Learning. Discriminative and Generative (Kluwer, Dordecht, 2004)Google Scholar
  17. 17.
    K. Murphy, The Machine Learning: A Probabilistic Perspective (MIT Press, Cambridge, 2012)Google Scholar
  18. 18.
    A. Patera, G. Rozza, Reduced Basis Approximation and A Posteriori Error Estimation for Parametrized Partial Differential Equations. MIT Pappalardo Graduate Monographs in Mechanical Engineering (MIT, 2007)Google Scholar
  19. 19.
    B. Peherstorfer, D. Pflüger, H.-J. Bungartz, Clustering based on density estimation with sparse grids, in KI 2012: Advances in Artificial Intelligence, ed. by B. Glimm, A. Krüger. Lecture Notes in Computer Science, vol. 7526 (Springer, Berlin, 2012), pp. 131–142Google Scholar
  20. 20.
    D. Pflüger, Data Mining mit Dünnen Gittern. Diplomarbeit, IPVS, Universität Stuttgart, 2005Google Scholar
  21. 21.
    D. Pflüger, Spatially Adaptive Sparse Grids for High-Dimensional Problems (Verlag Dr. Hut, München, 2010)Google Scholar
  22. 22.
    D. Pflüger, Spatially adaptive refinement, in Sparse Grids and Applications, ed. by J. Garcke, M. Griebel. Lecture Notes in Computational Science and Engineering, vol. 88 (Springer, Berlin, 2013)Google Scholar
  23. 23.
    D. Pflüger, B. Peherstorfer, H.-J. Bungartz, Spatially adaptive sparse grids for high-dimensional data-driven problems. J. Complex. 26(5), 508–522 (2010)CrossRefzbMATHGoogle Scholar
  24. 24.
    A. Uzilov, J. Keegan, D. Mathews, Detection of non-coding RNAs on the basis of predicted secondary structure formation free energy change. BMC Bioinformatics 7(1), 173 (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Benjamin Peherstorfer
    • 1
    Email author
  • Fabian Franzelin
    • 2
  • Dirk Pflüger
    • 2
  • Hans-Joachim Bungartz
    • 1
  1. 1.Department of InformaticsTechnische Universität MünchenGarchingGermany
  2. 2.Institute for Parallel and Distributed SystemsUniversity of StuttgartStuttgartGermany

Personalised recommendations