HODE: Hidden One-Dependence Estimator

  • M. Julia Flores
  • José A. Gámez
  • Ana M. Martínez
  • José M. Puerta
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5590)

Abstract

Among the several attempts to improve the Naive Bayes (NB) classifier, the Aggregating One-Dependence Estimators (AODE) has proved to be one of the most attractive, considering not only the low error it provides but also its efficiency. AODE estimates the corresponding parameters for every SPODE (Superparent-One-Dependence Estimators) using each attribute of the database as the superparent, and uniformly averages them all. Nevertheless, AODE has properties that can be improved. Firstly, the need to store all the models constructed leads to a high demand on space and hence, to the impossibility of dealing with problems of high dimensionality; secondly, even though it is fast, the computational time required for the training and the classification time is quadratic in the number of attributes. This is specially significant in the classification time, as it is frequently carried out in real time. In this paper, we propose the HODE classifier as an alternative approach to AODE in order to alleviate its problems by estimating a new variable (the hidden variable) as a superparent besides the class, whose main objective is to gather all the dependences existing in the AODE models. The results obtained show that this new algorithm provides similar results in terms of accuracy with a reduction in classification time and space complexity.

Keywords

AODE SPODE ODE Bayesian Networks Bayesian Classifiers Classification 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Langley, P., Iba, W., Thompson, K.: An Analysis of Bayesian Classifiers. In: 10th national conference on artificial intelligence, pp. 223–228. AAAI Press, Menlo Park (1992)Google Scholar
  2. 2.
    Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Wiley & Sons Inc., Chichester (1973)MATHGoogle Scholar
  3. 3.
    Domingos, P., Pazzani, M.: Beyond independence: Conditions for the optimality of the simple Bayesian classifier. In: 13th International Conference on Machine Learning, pp. 105–112. Morgan Kaufmann, Italy (1996)Google Scholar
  4. 4.
    Langley, P., Sage, S.: Induction of selective Bayesian classifiers. In: 10th Conference on Uncertainty in Artificial Intelligence, pp. 399–406. Morgan Kaufmann, San Francisco (1994)Google Scholar
  5. 5.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. J. Mach. Learn. 29(2-3), 131–163 (1997)CrossRefMATHGoogle Scholar
  6. 6.
    Kohavi, R.: Scaling Up the Accuracy of Naive-Bayes Classifiers: a Decision-Tree Hybrid. In: 2nd International Conference on Knowledge Discovery and Data Mining, pp. 202–207 (1996)Google Scholar
  7. 7.
    Elkan, C.: Boosting and Naive Bayesian learning, Technical report, Dept. Computer Science and Eng., Univ. of California, San Diego (1997)Google Scholar
  8. 8.
    Webb, G.I., Boughton, J.R., Wang, Z.: Not So Naive Bayes: Aggregating One-Dependence Estimators. J. Mach. Learn. 58(1), 5–24 (2005)CrossRefMATHGoogle Scholar
  9. 9.
    Sahami, M.: Learning limited dependence Bayesian classifiers. In: 2nd International Conference on Knowledge Discovery in Databases, pp. 335–338. AAAI Press, Menlo Park (1996)Google Scholar
  10. 10.
    Yang, Y., Korb, K., Ting, K.-M., Webb, G.I.: Ensemble Selection for SuperParent-One-Dependence Estimators. In: Zhang, S., Jarvis, R. (eds.) AI 2005. LNCS, vol. 3809, pp. 102–112. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Zheng, F., Webb, G.I.: A Comparative Study of Semi-naive Bayes Methods in Classification Learning. In: Simoff, S.J., Williams, G.J., Galloway, J., Kolyshkina, I. (eds.) 4th Australasian Data Mining Conference (AusDM 2005), pp. 141–156. University of Technology, Sydney (2005)Google Scholar
  12. 12.
    Zheng, Z., Webb, G.I.: Lazy Learning of Bayesian Rules. J. Mach. Learn. 41(1), 53–84 (2000)CrossRefGoogle Scholar
  13. 13.
    Keogh, E., Pazzani, M.: Learning Augmented Bayesian Classifiers: A Comparison of Distribution-based and Classification-based Approaches. In: 7th Int’l Workshop on AI and Statistics, Florida, pp. 225–230 (1999)Google Scholar
  14. 14.
    Dempster, A., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. J. R. Stat. Soc. Ser. B-Stat. Methodol. 39, 1–38 (1977)MATHGoogle Scholar
  15. 15.
    Langseth, H., Nielsen, T.D.: Classification using hierarchical Naïve Bayes models. J. Mach. Learn. 63(2), 135–159 (2006)CrossRefMATHGoogle Scholar
  16. 16.
    Lowd, D., Domingos, P.: Naive Bayes models for probability estimation. In: 22nd international conference on Machine learning, pp. 529–536. ACM, Bonn (2005)Google Scholar
  17. 17.
    Cheeseman, P., Stutz, J.: Bayesian classification (AutoClass): theory and results. In: Advances in knowledge discovery and data mining, pp. 153–180. AAAI Press, Menlo Park (1996)Google Scholar
  18. 18.
    Akaike, H.: A Bayesian analysis of the minimum AIC procedure. Ann. Inst. Statist. Math. 30A, 9–14 (1978)CrossRefMATHGoogle Scholar
  19. 19.
    Collection of Datasets available from the Weka Official HomePage, University of Waikato, http://www.cs.waikato.ac.nz/ml/weka/

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • M. Julia Flores
    • 1
  • José A. Gámez
    • 1
  • Ana M. Martínez
    • 1
  • José M. Puerta
    • 1
  1. 1.Computing Systems Department, Intelligent Systems and Data Mining group, i3AUniversity of Castilla-La ManchaAlbaceteSpain

Personalised recommendations