Skip to main content

Induction of Modular Classification Rules by Information Entropy Based Rule Generation

Part of the Studies in Computational Intelligence book series (SCI,volume 623)

Abstract

Prism has been developed as a modular classification rule generator following the separate and conquer approach since 1987 due to the replicated sub-tree problem occurring in Top-Down Induction of Decision Trees (TDIDT). A series of experiments have been done to compare the performance between Prism and TDIDT which proved that Prism may generally provide a similar level of accuracy as TDIDT but with fewer rules and fewer terms per rule. In addition, Prism is generally more tolerant to noise with consistently better accuracy than TDIDT. However, the authors have identified through some experiments that Prism may also give rule sets which tend to underfit training sets in some cases. This paper introduces a new modular classification rule generator, which follows the separate and conquer approach, in order to avoid the problems which arise with Prism. In this paper, the authors review the Prism method and its advantages compared with TDIDT as well as its disadvantages that are overcome by a new method using Information Entropy Based Rule Generation (IEBRG). The authors also set up an experimental study on the performance of the new method in classification accuracy and computational efficiency. The method is also evaluated comparatively with Prism.

Keywords

  • Classification Accuracy
  • Classification Rule
  • Minority Class
  • Target Class
  • Average Entropy

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-27267-2_7
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-27267-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufman (1993)

    Google Scholar 

  2. Hunt, E.B., Stone, P.J., Marin, J.: Experiments in Induction. Academic Press, New York (1966)

    Google Scholar 

  3. Michalski, R.S.: On the quasi-minimal solution of the general covering problem. In: Proceedings of the Fifth International Symposium on Information Processing, Bled, Yugoslavia, pp. 125–128 (1969)

    Google Scholar 

  4. Bramer, M.A.: Principles of Data Mining. Springer, London (2007)

    MATH  Google Scholar 

  5. Cendrowska, J.: PRISM: an algorithm for inducing modular rules. Int. J. Man Mach. Stud. 27, 349–370 (1987)

    CrossRef  MATH  Google Scholar 

  6. Shannon, C.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)

    Google Scholar 

  7. Stahl, F., Bramer, M.: Induction of modular classification rules: using Jmax-pruning. In: 30th SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge (2011)

    Google Scholar 

  8. Stahl, F., Bramer, M.: Jmax-pruning: a facility for the information theoretic pruning of modular classification rules. Knowl.-Based Syst. 29(2012), 12–19 (2012)

    CrossRef  Google Scholar 

  9. Deng, X.: A Covering-Based Algorithm for Classification: PRISM. CS831: Knowledge Discover in Databases (2012)

    Google Scholar 

  10. Kerber, R.: Chimerge: discretization of numeric attributes. In: AAAI’92 Proceedings of the tenth national conference on Artificial intelligence, pp. 123–128 (1992)

    Google Scholar 

  11. Bramer, M.A.: Inducer: a public domain workbench for data mining. Int. J. Syst. Sci. 36, 909–919 (2005)

    CrossRef  MATH  Google Scholar 

  12. Stahl, F., Bramer, M.: Computationally efficient induction of classification rules with the PMCRI and J-PMCRI frameworks. Knowl. Based Syst. 35(2012), 49–63 (2012)

    CrossRef  Google Scholar 

  13. Bramer, M.A.: Automatic induction of classification rules from examples using N-Prism. In: Research and Development in Intelligent Systems XVI, pp. 99–121. Springer, Berlin (2000)

    Google Scholar 

  14. Blake, C.L., Merz, C.J.: UCI repository of machine learning databases. Technical Report, University of California, Irvine, Department of Information and Computer Sciences (1998)

    Google Scholar 

  15. Bramer, M.A.: Using J-pruning to reduce overfitting of classification rules in noisy domains. In: Proceedings of 13th International Conference on Database and Expert Systems Applications— DEXA 2002, Aix-en-Provence, France, 2–6 Sept 2002

    Google Scholar 

  16. Bramer, M.A.: Using J-pruning to reduce overfitting in classification trees. In: Research and Development in Intelligent Systems XVIII, pp. 25–38. Springer, Berlin (2002)

    Google Scholar 

  17. Smyth, P., Goodman, R.M.: Rule induction using information theory. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 159–176. AAAI Press (1991)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Han Liu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Liu, H., Gegov, A. (2016). Induction of Modular Classification Rules by Information Entropy Based Rule Generation. In: Sgurev, V., Yager, R., Kacprzyk, J., Jotsov, V. (eds) Innovative Issues in Intelligent Systems. Studies in Computational Intelligence, vol 623. Springer, Cham. https://doi.org/10.1007/978-3-319-27267-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27267-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27266-5

  • Online ISBN: 978-3-319-27267-2

  • eBook Packages: EngineeringEngineering (R0)