MC-Tree: Improving Bayesian Anytime Classification

  • Philipp Kranen
  • Stephan Günnemann
  • Sergej Fries
  • Thomas Seidl
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6187)

Abstract

In scientific databases large amounts of data are collected to create knowledge repositories for deriving new insights or planning further experiments. These databases can be used to train classifiers that later categorize new data tuples. However, the large amounts of data might yield a time consuming classification process, e.g. for nearest neighbors or kernel density estimators. Anytime classifiers bypass this drawback by being interruptible at any time while the quality of the result improves with higher time allowances. Interruptible classifiers are especially useful when newly arriving data has to be classified on demand, e.g. during a running experiment. A statistical approach to anytime classification has recently been proposed using Bayes classification on kernel density estimates.

In this paper we present a novel data structure called MC-Tree (Multi-Class Tree) that significantly improves Bayesian anytime classification. The tree stores a hierarchy of mixture densities that represent objects from several classes. Data transformations are used during tree construction to optimize the condition of the tree with respect to multiple classes. Anytime classification is achieved through novel query dependent model refinement approaches that take the entropy of the current mixture components into account. We show in experimental evaluation that the MC-Tree outperforms previous approaches in terms of anytime classification accuracy.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Andre, D., Stone, P.: Physiological data modeling contest In: ICML 2004 (2004), http://www.cs.utexas.edu/users/pstone/workshops/2004icml/
  2. 2.
    Bayes, T.: An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society 53, 370–418 (1763)Google Scholar
  3. 3.
    Bouckaert, R.: Naive Bayes Classifiers that Perform Well with Continuous Variables. In: AI (2004)Google Scholar
  4. 4.
    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. DMKD Journal 2(2), 121–167 (1998)Google Scholar
  5. 5.
    de Leeuw, J.: Applications of convex analysis to multidimensional scaling. In: Recent Developments in Statistics, pp. 133–146 (1977)Google Scholar
  6. 6.
    DeCoste, D.: Anytime interval-valued outputs for kernel machines: Fast support vector machine classification via distance geometry. In: ICML, pp. 99–106 (2002)Google Scholar
  7. 7.
    Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. Wiley, Chichester (2000)Google Scholar
  8. 8.
    Esmeir, S., Markovitch, S.: Anytime induction of decision trees: An iterative improvement approach. In: Proc. of the 21st AAAI (2006)Google Scholar
  9. 9.
    Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases. In: ACM KDD (1996)Google Scholar
  10. 10.
    Guttman, A.: R-trees: A dynamic index structure for spatial searching. In: SIGMOD, pp. 47–57 (1984)Google Scholar
  11. 11.
    Hettich, S., Bay, S.: The UCI KDD archive (1999), http://kdd.ics.uci.edu
  12. 12.
    John, G., Langley, P.: Estimating continuous distributions in bayesian classifiers. In: UAI. Morgan Kaufmann, San Francisco (1995)Google Scholar
  13. 13.
    Kranen, P., Assent, I., Baldauf, C., Seidl, T.: Self-adaptive anytime stream clustering. In: Proc. of the 9th IEEE ICDM (2009)Google Scholar
  14. 14.
    Kranen, P., Seidl, T.: Harnessing the strengths of anytime algorithms for constant data streams. DMKD Journal, ECML PKDD Special Issue 19(2), 245–260 (2009)Google Scholar
  15. 15.
    Kullback, S.: Information Theory and Statistics. Wiley, New York (1959)MATHGoogle Scholar
  16. 16.
    Lauritzen, S.: The EM algorithm for graphical association models with missing data. Comp. Statistics & Data Analysis 19, 191–201 (1995)MATHCrossRefGoogle Scholar
  17. 17.
    Patrick, E., Fischer, F.: A generalized k-nearest neighbor rule. Information and Control 16(2), 128–152 (1970)MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)Google Scholar
  19. 19.
    Seidl, T., Assent, I., Kranen, P., Krieger, R., Herrmann, J.: Indexing density models for incremental learning and anytime classification on data streams. In: EDBT, pp. 311–322 (2009)Google Scholar
  20. 20.
    Silverman, B.: Density Estimation for Statistics and Data Analysis. Chapman & Hall/CRC, Boca Raton (1986)MATHGoogle Scholar
  21. 21.
    Ueno, K., Xi, X., Keogh, E.J., Lee, D.-J.: Anytime classification using the nearest neighbor algorithm with applications to stream mining. In: ICDM (2006)Google Scholar
  22. 22.
    Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)MATHGoogle Scholar
  23. 23.
    Zilberstein, S.: Using anytime algorithms in intelligent systems. The AI magazine 17(3), 73–83 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Philipp Kranen
    • 1
  • Stephan Günnemann
    • 1
  • Sergej Fries
    • 1
  • Thomas Seidl
    • 1
  1. 1.Data management and data exploration groupRWTH Aachen UniversityGermany

Personalised recommendations