Converting a Naive Bayes Model into a Set of Rules

  • Bartłomiej Śnieżyński
Part of the Advances in Soft Computing book series (AINSC, volume 35)


A knowledge representation based on the probability theory is currently the most popular way of handling uncertainty. However, rule based systems are still popular. Their advantage is that rules are usually more easy to interpret than probabilistic models. A conversion method would allow to exploit advantages of both techniques. In this paper an algorithm that converts Naive Bayes models into rule sets is proposed. Preliminary experimental results show that rules generated from Naive Bayes models are compact and accuracy of such rule-based classifiers are relatively high.


Decision Rule Bayesian Network Medical Informatics Default Rule Bayesian Belief Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    1. E.G. Buchanan and H. Shortliffe. Rule-based expert systems: The MYCIN experiments of the Stanford heuristic programming project. Addison-Wesley, 1984.Google Scholar
  2. 2.
    2. C.L. Blake D.J. Newman, S. Hettich and C.J. Merz. UCI repository of machine learning databases, 1998.Google Scholar
  3. 3.
    3. M.J. Druzdzel. A development environment for graphical decision-analytic models. In Proc. of the 1999 Annual Symposium of the American Medical Informatics Association (AMIA-1999), page 1206, Washington, B.C., 1999.Google Scholar
  4. 4.
    4. N. Friedman, D. Geiger, and M. Goldszmidt. Bayesian network classifiers. Machine Learning, 29(2–3):131–163, 1997.zbMATHCrossRefGoogle Scholar
  5. 5.
    5. D. Heckerman. Probabilistic interpretation for MYCIN's uncertainty factors, pages 167–196. North-Holland, 1986.Google Scholar
  6. 6.
    6. D.E. Heckerman. An empirical comparison of three inference methods. In Proceedings of the Fourth Workshop on Uncertainty in Artificial Intelligence, pages 158–169. Association for Uncertainty in Artificial Intelligence, Mountain View, CA, 1988.Google Scholar
  7. 7.
    7. D. Roller and A. Pfeffer. Probabilistic frame-based systems. In Proc. of 15th National Conference on Artificial Intelligence AAAI-98, pages 580–587, 1998.Google Scholar
  8. 8.
    8. M. Korver and P. Lucas. Converting a rule-based expert system into a belief network. Medical Informatics, 18(3):219–241, 1993.Google Scholar
  9. 9.
    9. P.J.F. Lucas. Certainty-factor-like structures in bayesian belief networks. Knowl.-Based Syst, 14(7):327–335, 2001.CrossRefGoogle Scholar
  10. 10.
    10. P.J.F. Lucas and A.R. Janssens. Development and validation of hepar, an expert system for the diagnosis of disorders of the liver and biliary tract. Medical Informatics, 16:259–270, 1991.CrossRefGoogle Scholar
  11. 11.
    11. R. S. Michalski and I. Imam. Learning problem-oriented decision structures from decision rules: The aqdt-2 system. In Methodology for Intelligent Systems of the 8th International Symposium on Methodology for Intelligent Systems (ISMIS-94), volume 869 of Lecture Notes in Artificial Intelligence, pages 416–426. Springer, 1994.Google Scholar
  12. 12.
    12. B. Middleton, M. Shwe, Heckerman, M. Henrion D. E., E. J. Horvitz, H. Lehmann, and G. F. Cooper. Probabilistic diagnosis using a reformulation of the internist-1/qmr knowledge base ii: Evaluation of diagnostic performance. Methods of Information in Medicine, 30:256–267, 1991.Google Scholar
  13. 13.
    13. A. Newell and H.A. Simon. Human Problem Solving. Prentice-Hall, 1972.Google Scholar
  14. 14.
    14. A. Onisko, P. Lucas, and M.J. Druzdzel. Comparison of rule-based and Bayesian network approaches in medical diagnostic systems. Lecture Notes in Computer Science, 2101:283+, 2001.CrossRefGoogle Scholar
  15. 15.
    15. D. Poole. Probabilistic horn abduction and bayesian networks. Artificial Intelligence, 64(1):81–129, 1993.zbMATHCrossRefGoogle Scholar
  16. 16.
    16. J.R. Quinlan. C4-5: Programs for Machine Learning. Morgan Kaufmann, 1993.Google Scholar
  17. 17.
    17. M. Shwe, B. Middleton, D. E. Heckerman, M. Henrion, E. J. Horvitz, H. Lehmann, and G. F. Cooper. Probabilistic diagnosis using a reformulation of the internist-1/qmr knowledge base i: Probabilistic model and inference algorithms. Methods of Information in Medicine, 30:241–255, 1991.Google Scholar
  18. 18.
    18. B. Sniezynski. Choice of a knowledge representation method for learning classifiers in medical domains. Journal of Medical Informatics and Technologies, 6, 2005.Google Scholar

Copyright information

© Springer 2006

Authors and Affiliations

  • Bartłomiej Śnieżyński
    • 1
  1. 1.Institute of Computer ScienceAGH University of Science and TechnologyKrakówPoland

Personalised recommendations