Random Multiclass Classification: Generalizing Random Forests to Random MNL and Random NB

  • Anita Prinzie
  • Dirk Van den Poel
Conference paper

DOI: 10.1007/978-3-540-74469-6_35

Part of the Lecture Notes in Computer Science book series (LNCS, volume 4653)
Cite this paper as:
Prinzie A., Van den Poel D. (2007) Random Multiclass Classification: Generalizing Random Forests to Random MNL and Random NB. In: Wagner R., Revell N., Pernul G. (eds) Database and Expert Systems Applications. DEXA 2007. Lecture Notes in Computer Science, vol 4653. Springer, Berlin, Heidelberg

Abstract

Random Forests (RF) is a successful classifier exhibiting performance comparable to Adaboost, but is more robust. The exploitation of two sources of randomness, random inputs (bagging) and random features, make RF accurate classifiers in several domains. We hypothesize that methods other than classification or regression trees could also benefit from injecting randomness. This paper generalizes the RF framework to other multiclass classification algorithms like the well-established MultiNomial Logit (MNL) and Naive Bayes (NB). We propose Random MNL (RMNL) as a new bagged classifier combining a forest of MNLs estimated with randomly selected features. Analogously, we introduce Random Naive Bayes (RNB). We benchmark the predictive performance of RF, RMNL and RNB against state-of-the-art SVM classifiers. RF, RMNL and RNB outperform SVM. Moreover, generalizing RF seems promising as reflected by the improved predictive performance of RMNL.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Anita Prinzie
    • 1
  • Dirk Van den Poel
    • 1
  1. 1.Department of Marketing, Ghent University, Tweekerkenstraat 2, 9000 GhentBelgium

Personalised recommendations