Article

Machine Learning

, Volume 58, Issue 1, pp 5-24

Not So Naive Bayes: Aggregating One-Dependence Estimators

  • Geoffrey I. WebbAffiliated withSchool of Computer Science and Software Engineering, Monash University Email author 
  • , Janice R. BoughtonAffiliated withSchool of Computer Science and Software Engineering, Monash University
  • , Zhihai WangAffiliated withSchool of Computer Science and Software Engineering, Monash University

Abstract

Of numerous proposals to improve the accuracy of naive Bayes by weakening its attribute independence assumption, both LBR and Super-Parent TAN have demonstrated remarkable error performance. However, both techniques obtain this outcome at a considerable computational cost. We present a new approach to weakening the attribute independence assumption by averaging all of a constrained class of classifiers. In extensive experiments this technique delivers comparable prediction accuracy to LBR and Super-Parent TAN with substantially improved computational efficiency at test time relative to the former and at training time relative to the latter. The new algorithm is shown to have low variance and is suited to incremental learning.

naive Bayes semi-naive Bayes attribute independence assumption probabilistic prediction