Skip to main content
SpringerLink
Log in

Increasing the accuracy of incremental naive bayes classifier using instance based learning

  • Regular Paper
  • Intelligent and Information Systems
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

Abstract

Along with the increase of data and information, incremental learning ability turns out to be more and more important for machine learning approaches. The online algorithms try not to remember irrelevant information instead of synthesizing all available information (as opposed to classic batch learning algorithms). In this study, we attempted to increase the prediction accuracy of an incremental version of Naive Bayes model by integrating instance based learning. We performed a large-scale comparison of the proposed method with other state-of-the-art algorithms on several datasets and the proposed method produce better accuracy in most cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P. Auer and M. Warmuth, “Tracking the best disjunction,” Machine Learning, vol. 32, no. 2, pp. 127–150, 1998.

    Article  MATH  Google Scholar 

  2. F. Chu and C. Zaniolo, “Fast and light boosting for adaptive mining of data streams,” Lecture Notes in Computer Science, vol. 3056, pp. 282–292, 2004.

    Article  Google Scholar 

  3. J. G. Cleary and L. E. Trigg, “K*: an instance-based learner using an entropic distance measure,” Proc. of the 12th International Conference on Machine Learning, pp. 108–114, 1995.

  4. W. Cohen, “fast effective rule induction,” Proc. of Int Conf. of ML-95, pp. 115–123, 1995.

  5. P. Domingos and M. Pazzani, “On the optimality of the simple Bayesian classifier under zero-one loss,” Machine Learning, vol. 29, no. 2–3, pp. 103–130, 1997.

    Article  MATH  Google Scholar 

  6. W. Fan, S. Stolfo, and J. Zhang, “The application of AdaBoost for distributed, scalable and online learning,” Proc. of Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, pp. 362–366, 1999.

  7. F. Fdez-Riverola, E. L. Iglesias, F. Díaz, J. R. Méndez, and J. M. Corchado, “Applying lazy learning algorithms to tackle concept drift in spam filtering,” Expert Systems with Applications, vol. 33, no. 1, pp. 36–48, 2007.

    Article  Google Scholar 

  8. A. Fern and R. Givan, “Online ensemble learning: An empirical study,” Proc. of the 17th International Conference on ML, pp. 279–286, 2000.

  9. A. Frank and A. Asuncion, UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, 2010.

    Google Scholar 

  10. Y. Freund and R. Schapire, “Large margin classification using the perceptron algorithm,” Machine Learning, vol. 37, no. 3, pp. 277–296, 1999.

    Article  MATH  Google Scholar 

  11. A. Gangardiwala and R. Polikar, “Dynamically weighted majority voting for incremental learning and comparison of three boosting based approaches,” Proc. of Joint Conf. on Neural Networks, pp. 1131–1136, 2005.

  12. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, “The WEKA data mining software: an update,” SIGKDD Explorations, vol. 11, no. 1, pp. 10–18, 2009.

    Article  Google Scholar 

  13. L. I. Kuncheva, “Classifier ensembles for changing environments,” Lecture Notes in Computer Science, vol. 3077, pp. 1–15, 2004.

    Article  Google Scholar 

  14. L. Kai and H.-K. Huang, “Incremental learning proximal support vector machine classifiers,” Proc. of International Conference on Machine Learning and Cybernetics, pp. 1635–1637, 2002.

  15. J. Lee, W. Chung, E. Kim, and S. Kim, “A new genetic approach for structure learning of Bayesian networks: matrix genetic algorithm,” International Journal of Control, Automation and Systems, vol. 8, no. 2, pp 398–407, 2010.

    Article  Google Scholar 

  16. Z. Liang and Y. Li, “Incremental support vector machine learning in the primal and applications,” Neurocomputing, vol. 72, no. 10–12, pp. 2249–2258, 2009.

    Article  Google Scholar 

  17. N. Littlestone and M. Warmuth, “The weighted majority algorithm,” Information and Computation, vol. 108, pp. 212–261, 1994.

    Article  MathSciNet  MATH  Google Scholar 

  18. L. Jing, L. Xue, and Z. Weicai, “Ambiguous decision trees for mining concept-drifting data streams,” Pattern Recognition Letters, vol. 30, no. 15, pp. 1347–1355, 2009.

    Article  Google Scholar 

  19. N. C. Oza and S. Russell, “Online bagging and boosting,” Proc. of Artificial Intelligence and Statistics 2001, pp. 105–112, 2001.

  20. J. R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann, San Francisco, 1993.

    Google Scholar 

  21. D. Saad, Online Learning in Neural Networks, London, Cambridge University Press, 1998.

    Google Scholar 

  22. M. Sahami, “Learning limited dependence Bayesian classifiers,” Proc. of the 2nd Int. Conf. on Knowledge Discovery in Databases, pp. 335–338, 1996.

  23. R. Sylvain, Nearest Neighbor with Generalization, Christchurch, New Zealand, 2002.

    Google Scholar 

  24. C.-J. Tsai, C.-I. Lee, and W.-P. Yang, “Mining decision rules on data streams in the presence of concept drifts,” Expert Systems with Applications, vol. 36, no. 2, pp. 1164–1178, 2009.

    Article  Google Scholar 

  25. A. Tsymbal, M. Pechenizkiy, P. Cunningham, and S. Puuronen, “Dynamic integration of classifiers for handling concept drift,” Information Fusion, vol. 9, no. 1, pp. 56–68, 2008.

    Article  Google Scholar 

  26. P. Utgoff, N. Berkman, and J. Clouse, “Decision tree induction based on efficient tree restructuring,” Machine Learning, vol. 29, no. 1, pp. 5–44, 1997.

    Article  MATH  Google Scholar 

  27. G. I. Webb, J. R. Boughton, and Z. Wang, “Not so naive bayes: aggregating one-dependence estimators,” Machine Learning, vol. 58, pp. 5–24, 2005.

    Article  MATH  Google Scholar 

  28. G. Widmer and M. Kubat, “Learning in the presence of concept drift and hidden contexts,” Machine Learning, vol. 23, pp. 69–101, 1996.

    Google Scholar 

  29. I. Witten, E. Frank, and M. Hall, Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann, 2011.

  30. H.-G. Yeom, S.-M. Park, J. Park, and K.-B. Sim, “Superiority demonstration of variance-considered machines by comparing error rate with support vector machines,” Int. Journal of Control, Automation, and Systems, vol. 9, no. 3, pp. 595–600, 2011.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Educational Software Development Laboratory of the Department of Mathematics, University of Patras, Rio, 26504, Greece

    Sotiris Kotsiantis

Authors
  1. Sotiris Kotsiantis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sotiris Kotsiantis.

Additional information

Recommended by Editorial Board member Yuan Fang Zheng under the direction of Editor Myotaeg Lim.

Sotiris Kotsiantis received his bachelor in mathematics in 1999, a Master degree in 2001 and a Ph.D. degree in computer science in 2005 from the University of Patras, Greece. His research interests are mainly in the field of data mining and machine learning. He has a lot of publications to his credit in international journals and conferences.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kotsiantis, S. Increasing the accuracy of incremental naive bayes classifier using instance based learning. Int. J. Control Autom. Syst. 11, 159–166 (2013). https://doi.org/10.1007/s12555-011-0099-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-011-0099-1

Keywords

Search

Navigation