Advertisement

Data Mining and Knowledge Discovery

, Volume 19, Issue 3, pp 293–319 | Cite as

FURIA: an algorithm for unordered fuzzy rule induction

  • Jens Hühn
  • Eyke Hüllermeier
Article

Abstract

This paper introduces a novel fuzzy rule-based classification method called FURIA, which is short for Fuzzy Unordered Rule Induction Algorithm. FURIA extends the well-known RIPPER algorithm, a state-of-the-art rule learner, while preserving its advantages, such as simple and comprehensible rule sets. In addition, it includes a number of modifications and extensions. In particular, FURIA learns fuzzy rules instead of conventional rules and unordered rule sets instead of rule lists. Moreover, to deal with uncovered examples, it makes use of an efficient rule stretching method. Experimental results show that FURIA significantly outperforms the original RIPPER, as well as other classifiers such as C4.5, in terms of classification accuracy.

Keywords

Classification Rule learning Fuzzy logic Rule stretching 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aha D, Kibler D, Albert M (1991) Instance-based learning algorithms. Mach Learn 6(1): 37–66Google Scholar
  2. Alcalá-Fernandez J, Sánchez L, García S, del Jesus M, Ventura S, Garrell J, Otero J, Romero C, Bacardit J, Rivas V, Fernández J, Herrera F (2009) Keel: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3): 307–318CrossRefGoogle Scholar
  3. Asuncion A, Newman D (2007) UCI machine learning repository. http://archive.ics.uci.edu/ml/index.html. Accessed 20 Aug 2007
  4. Barker D (2007) Dataset: pasture production. http://weka.sourceforge.net/wiki/index.php/Datasets. Accessed 20 Oct 2007
  5. Boström H (2004) Pruning and exclusion criteria for unordered incremental reduced error pruning. Proceedings of the workshop on advances in rule learning, ECML, pp 17–29Google Scholar
  6. Bulloch B (2007) Dataset: eucalyptus soil conservation. http://weka.sourceforge.net/wiki/index.php/Datasets. Accessed 20 Oct 2007
  7. Chi Z, Wu J, Yan H (1995) Handwritten numeral recognition using self-organizing maps and fuzzy rules. Pattern Recogn 28(1): 59–66CrossRefGoogle Scholar
  8. Chi Z, Yan H, Pham T (1996) Fuzzy algorithms: with applications to image processing and pattern recognition. World Scientific Publishing Co., Inc., River EdgeGoogle Scholar
  9. Cloete I, Van Zyl J (2006) Fuzzy rule induction in a set covering framework. IEEE Trans Fuzzy Syst 14(1): 93–110CrossRefGoogle Scholar
  10. Cohen W (1995) Fast effective rule induction. In: Prieditis A, Russell S (eds) Proceedings of the 12th international conference on machine learning, ICML. Morgan Kaufmann, Tahoe City, pp 115–123Google Scholar
  11. Cordon O, Gomide F, Herrera F, Hoffmann F, Magdalena L (2004) Ten years of genetic fuzzy systems: current framework and new trends. Fuzzy Sets Syst 141(1): 5–31CrossRefMathSciNetMATHGoogle Scholar
  12. del Jesus M, Hoffmann F, Navascues L, Sd́fnchez L (2004) Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms. IEEE Trans Fuzzy Syst 12(3): 296–308CrossRefGoogle Scholar
  13. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7: 1–30MathSciNetGoogle Scholar
  14. Domingos P (1995) Rule induction and instance-based learning: a unified approach. In: Proceedings of the fourteenth international joint conference on artificial intelligence, IJCAI. Morgan Kaufmann, Montral, pp 1226–1232Google Scholar
  15. Drobics M, Bodenhofer U, Klement E (2003) FS-FOIL: an inductive learning method for extracting interpretable fuzzy descriptions. Int J Approx Reason 32(2–3): 131–152CrossRefMATHGoogle Scholar
  16. Eineborg M, Boström H (2001) Classifying uncovered examples by rule stretching. In: ILP ’01: Proceedings of the 11th international conference on inductive logic programming. Springer-Verlag, London, pp 41–50Google Scholar
  17. Fawcett T (2008) Prie: a system for generating rulelists to maximize roc performance. Data Min Knowl Discov 17(2): 207–224CrossRefGoogle Scholar
  18. Fernández A, García S, Herrera F, del Jesus M (2007) An analysis of the rule weights and fuzzy reasoning methods for linguistic rule based classification systems applied to problems with highly imbalanced data sets. In: Applications of fuzzy sets theory, vol 4578, Lecture notes in computer science. Springer, Berlin/Heidelberg, pp 170–178Google Scholar
  19. Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200): 675–701CrossRefGoogle Scholar
  20. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1): 86–92CrossRefMATHGoogle Scholar
  21. Fürnkranz J (1999) Separate-and-conquer rule learning. Artif Intell Rev 13(1): 3–54CrossRefMATHGoogle Scholar
  22. Fürnkranz J, Widmer G (1994) Incremental reduced error pruning. In: Proceedings of the 11th international conference on machine learning, ICML, pp 70–77Google Scholar
  23. Gonzalez A (1999) Slave: a genetic learning system based on an iterative approach. IEEE Trans Fuzzy Syst 7(2): 176–191CrossRefGoogle Scholar
  24. Gonzalez A (2001) Selection of relevant features in a fuzzy genetic learning algorithm. IEEE Trans Syst Man Cyber B 31(3): 417–425CrossRefGoogle Scholar
  25. Guillaume S (2001) Defining fuzzy inference systems from data: an interpretability-oriented review. IEEE Trans Fuzzy Syst 9(3): 426–443CrossRefMathSciNetGoogle Scholar
  26. Harvey W (2007) Dataset: squash harvest stored/unstored. http://weka.sourceforge.net/wiki/index.php/Datasets. Accessed on 20 Oct 2007
  27. Hendrickx I, van den Bosch A (2005) Hybrid algorithms for instance-based classification. In: Gama J, Camacho R, Brazdil P, Jorge A, Torgo L (eds) Proceedings of the sixteenth European conference on machine learning, ECML. Springer, Berlin, pp 158–169Google Scholar
  28. Hühn J, Hüllermeier E (2009) FR3: a fuzzy rule learner for inducing reliable classifiers. IEEE Trans Fuzzy Syst 17(1): 138–149CrossRefGoogle Scholar
  29. Hüllermeier E (2005) Fuzzy sets in machine learning and data mining: status and prospects. Fuzzy Sets Syst 156(3): 387–406CrossRefGoogle Scholar
  30. Iman R, Davenport J (1980) Approximations of the critical region of the friedman statistic. Commun Stat 9(6): 571–595CrossRefGoogle Scholar
  31. Ishibuchi H, Nakashima T (2001) Effect of rule weights in fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 9(4): 506–515CrossRefGoogle Scholar
  32. Ishibuchi H, Yamamoto T (2003) Performance evaluation of three-objective genetic rule selection. In: The 12th IEEE international conference on fuzzy systems, vol 1, pp 149–154Google Scholar
  33. Ishibuchi H, Yamamoto T (2005) Rule weight specification in fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 13(4): 428–436CrossRefGoogle Scholar
  34. Juang C, Chiu S, Chang S (2007) A self-organizing ts-type fuzzy network with support vector learning and its application to classification problems. IEEE Trans Fuzzy Syst 15(5): 998–1008CrossRefGoogle Scholar
  35. Kamal M, Pazzani M (1993) Hydra: a noise-tolerant relational concept learning algorithm. In: Proceedings of the thirteenth international joint conference on artificial intelligence, IJCAI. Morgan Kaufmann, Chambry, pp 1064–1071Google Scholar
  36. Kearns M (1988) Thoughts on hypothesis boosting. ML class projectGoogle Scholar
  37. Meyer M, Vlachos P (2007) Statlib.http://lib.stat.cmu.edu/
  38. Mitra S, Hayashi Y (2000) Neuro-fuzzy rule generation: survey in soft computing framework. IEEE Trans Neural Netw 11(3): 748–768CrossRefGoogle Scholar
  39. Nauck D, Klawonn F, Kruse R (1997) Foundations of neuro-fuzzy systems. Chichester, WileyGoogle Scholar
  40. Nemenyi P (1963) Distribution-free multiple comparisons. PhD thesis, Princeton UniversityGoogle Scholar
  41. Newman D (1939) The distribution of range in samples from a normal population expressed in terms of an independent estimate of standard deviation. Biometrika 31: 20–30MathSciNetMATHGoogle Scholar
  42. Prade H, Richard G, Serrurier M (2003) Enriching relational learning with fuzzy predicates. In: Proceedings of PKDD–03 European conference on principles and practice of knowledge discovery in databases, pp 399–410Google Scholar
  43. Press W, Flannery B, Teukolsky S, Vetterling W (1992) Numerical recipes in FORTRAN: the art of scientific computing, 2nd edn. Cambridge University PressGoogle Scholar
  44. Quinlan J (1990) Learning logical definitions from relations. Mach Learn 5(3): 239–266Google Scholar
  45. Quinlan J (1993) C4.5: programs for machine learning. Morgan Kaufmann, San FranciscoGoogle Scholar
  46. Quinlan J (1995) MDL and categorial theories (continued). In: Proceedings of the 12th international conference on machine learning, ICML. Morgan Kaufmann, Lake Tahoe, pp 464–470Google Scholar
  47. Quinlan J, Cameron-Jones R (1993) Foil: a midterm report. In: Proceedings of the 6th European conference on machine learning, ECML. Springer-Verlag, London, pp 3–20Google Scholar
  48. Serrurier M, Prade H (2007) Introducing possibilistic logic in ILP for dealing with exceptions. Artif Intell 171(16–17): 939–950CrossRefMathSciNetMATHGoogle Scholar
  49. Wang L, Mendel J (1992) Generating fuzzy rules by learning from examples. Trans Syst Man Cyber 22(6): 1414–1427CrossRefMathSciNetGoogle Scholar
  50. Wang T, Li Z, Yan Y, Chen H (2007) A survey of fuzzy decision tree classifier methodology. In: Proceedings of the second international conference of fuzzy information and engineering, vol 40 of advances in soft computing. Springer-Verlag, Berlin/HeidelbergGoogle Scholar
  51. Witten I, Frank E (2005) Data mining: practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San FranciscoMATHGoogle Scholar
  52. Zolghadri M, Mansoori E (2007) Weighting fuzzy classification rules using receiver operating characteristics (roc) analysis. Inf Sci 177(11): 2296–2307CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Mathematics and Computer SciencePhilipps-Universität MarburgMarburgGermany

Personalised recommendations