MC-Tree: Improving Bayesian Anytime Classification
In scientific databases large amounts of data are collected to create knowledge repositories for deriving new insights or planning further experiments. These databases can be used to train classifiers that later categorize new data tuples. However, the large amounts of data might yield a time consuming classification process, e.g. for nearest neighbors or kernel density estimators. Anytime classifiers bypass this drawback by being interruptible at any time while the quality of the result improves with higher time allowances. Interruptible classifiers are especially useful when newly arriving data has to be classified on demand, e.g. during a running experiment. A statistical approach to anytime classification has recently been proposed using Bayes classification on kernel density estimates.
In this paper we present a novel data structure called MC-Tree (Multi-Class Tree) that significantly improves Bayesian anytime classification. The tree stores a hierarchy of mixture densities that represent objects from several classes. Data transformations are used during tree construction to optimize the condition of the tree with respect to multiple classes. Anytime classification is achieved through novel query dependent model refinement approaches that take the entropy of the current mixture components into account. We show in experimental evaluation that the MC-Tree outperforms previous approaches in terms of anytime classification accuracy.
KeywordsSupport Vector Machine Class Label Kernel Density Near Neighbor Kernel Density Estimation
Unable to display preview. Download preview PDF.
- 1.Andre, D., Stone, P.: Physiological data modeling contest In: ICML 2004 (2004), http://www.cs.utexas.edu/users/pstone/workshops/2004icml/
- 2.Bayes, T.: An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society 53, 370–418 (1763)Google Scholar
- 3.Bouckaert, R.: Naive Bayes Classifiers that Perform Well with Continuous Variables. In: AI (2004)Google Scholar
- 4.Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. DMKD Journal 2(2), 121–167 (1998)Google Scholar
- 5.de Leeuw, J.: Applications of convex analysis to multidimensional scaling. In: Recent Developments in Statistics, pp. 133–146 (1977)Google Scholar
- 6.DeCoste, D.: Anytime interval-valued outputs for kernel machines: Fast support vector machine classification via distance geometry. In: ICML, pp. 99–106 (2002)Google Scholar
- 7.Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. Wiley, Chichester (2000)Google Scholar
- 8.Esmeir, S., Markovitch, S.: Anytime induction of decision trees: An iterative improvement approach. In: Proc. of the 21st AAAI (2006)Google Scholar
- 9.Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases. In: ACM KDD (1996)Google Scholar
- 10.Guttman, A.: R-trees: A dynamic index structure for spatial searching. In: SIGMOD, pp. 47–57 (1984)Google Scholar
- 11.Hettich, S., Bay, S.: The UCI KDD archive (1999), http://kdd.ics.uci.edu
- 12.John, G., Langley, P.: Estimating continuous distributions in bayesian classifiers. In: UAI. Morgan Kaufmann, San Francisco (1995)Google Scholar
- 13.Kranen, P., Assent, I., Baldauf, C., Seidl, T.: Self-adaptive anytime stream clustering. In: Proc. of the 9th IEEE ICDM (2009)Google Scholar
- 14.Kranen, P., Seidl, T.: Harnessing the strengths of anytime algorithms for constant data streams. DMKD Journal, ECML PKDD Special Issue 19(2), 245–260 (2009)Google Scholar
- 18.Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)Google Scholar
- 19.Seidl, T., Assent, I., Kranen, P., Krieger, R., Herrmann, J.: Indexing density models for incremental learning and anytime classification on data streams. In: EDBT, pp. 311–322 (2009)Google Scholar
- 21.Ueno, K., Xi, X., Keogh, E.J., Lee, D.-J.: Anytime classification using the nearest neighbor algorithm with applications to stream mining. In: ICDM (2006)Google Scholar
- 23.Zilberstein, S.: Using anytime algorithms in intelligent systems. The AI magazine 17(3), 73–83 (1996)Google Scholar