HODE: Hidden One-Dependence Estimator
Among the several attempts to improve the Naive Bayes (NB) classifier, the Aggregating One-Dependence Estimators (AODE) has proved to be one of the most attractive, considering not only the low error it provides but also its efficiency. AODE estimates the corresponding parameters for every SPODE (Superparent-One-Dependence Estimators) using each attribute of the database as the superparent, and uniformly averages them all. Nevertheless, AODE has properties that can be improved. Firstly, the need to store all the models constructed leads to a high demand on space and hence, to the impossibility of dealing with problems of high dimensionality; secondly, even though it is fast, the computational time required for the training and the classification time is quadratic in the number of attributes. This is specially significant in the classification time, as it is frequently carried out in real time. In this paper, we propose the HODE classifier as an alternative approach to AODE in order to alleviate its problems by estimating a new variable (the hidden variable) as a superparent besides the class, whose main objective is to gather all the dependences existing in the AODE models. The results obtained show that this new algorithm provides similar results in terms of accuracy with a reduction in classification time and space complexity.
KeywordsAODE SPODE ODE Bayesian Networks Bayesian Classifiers Classification
Unable to display preview. Download preview PDF.
- 1.Langley, P., Iba, W., Thompson, K.: An Analysis of Bayesian Classifiers. In: 10th national conference on artificial intelligence, pp. 223–228. AAAI Press, Menlo Park (1992)Google Scholar
- 3.Domingos, P., Pazzani, M.: Beyond independence: Conditions for the optimality of the simple Bayesian classifier. In: 13th International Conference on Machine Learning, pp. 105–112. Morgan Kaufmann, Italy (1996)Google Scholar
- 4.Langley, P., Sage, S.: Induction of selective Bayesian classifiers. In: 10th Conference on Uncertainty in Artificial Intelligence, pp. 399–406. Morgan Kaufmann, San Francisco (1994)Google Scholar
- 6.Kohavi, R.: Scaling Up the Accuracy of Naive-Bayes Classifiers: a Decision-Tree Hybrid. In: 2nd International Conference on Knowledge Discovery and Data Mining, pp. 202–207 (1996)Google Scholar
- 7.Elkan, C.: Boosting and Naive Bayesian learning, Technical report, Dept. Computer Science and Eng., Univ. of California, San Diego (1997)Google Scholar
- 9.Sahami, M.: Learning limited dependence Bayesian classifiers. In: 2nd International Conference on Knowledge Discovery in Databases, pp. 335–338. AAAI Press, Menlo Park (1996)Google Scholar
- 11.Zheng, F., Webb, G.I.: A Comparative Study of Semi-naive Bayes Methods in Classification Learning. In: Simoff, S.J., Williams, G.J., Galloway, J., Kolyshkina, I. (eds.) 4th Australasian Data Mining Conference (AusDM 2005), pp. 141–156. University of Technology, Sydney (2005)Google Scholar
- 13.Keogh, E., Pazzani, M.: Learning Augmented Bayesian Classifiers: A Comparison of Distribution-based and Classification-based Approaches. In: 7th Int’l Workshop on AI and Statistics, Florida, pp. 225–230 (1999)Google Scholar
- 16.Lowd, D., Domingos, P.: Naive Bayes models for probability estimation. In: 22nd international conference on Machine learning, pp. 529–536. ACM, Bonn (2005)Google Scholar
- 17.Cheeseman, P., Stutz, J.: Bayesian classification (AutoClass): theory and results. In: Advances in knowledge discovery and data mining, pp. 153–180. AAAI Press, Menlo Park (1996)Google Scholar
- 19.Collection of Datasets available from the Weka Official HomePage, University of Waikato, http://www.cs.waikato.ac.nz/ml/weka/