Abstract
This chapter focuses on one particular class of data mining methodologies that expresses the mined knowledge in the form of fuzzy If-Then rules or fuzzy decision trees that can be easily understood by a human. Past studies on generating fuzzy If-Then rules (mostly from exemplar crisp data and a few from exemplar fuzzy data) are grouped into six major categories: grid partitioning, fuzzy clustering, genetic algorithms, neural networks, hybrid methods, and others. The representative method in each category is detailed. The latest improvements and advancements in each category are also reviewed. Similarly, past studies on generating fuzzy decision trees (from exemplar nominal and/or numeric data as well as from exemplar fuzzy data) are surveyed. The essence of each method is presented. Moreover, we discuss selected studies that address most of the necessary conditions for a fuzzy model to be interpretable and highlight areas for future studies. To give an idea of where fuzzy modeling methods have been applied, major application areas are also summarized.
Triantaphyllou, E. and G. Felici (Eds.), Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques, Massive Computing Series, Springer, Heidelberg, Germany, pp. 495–550, 2006.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abe, S. and Lan, M.-S., “A method for fuzzy rules extraction directly from numerical data and its application to pattern recognition,” IEEE Trans. Fuzzy Systems, 3(1), 1995, 18–28.
Azeem, M. F., Hanmandlu, M., and Ahmad, N., “Generalization of adaptive neuro-fuzzy inference systems,” IEEE Trans. Neural Networks, 11(6), 2000, 1332–1346.
Behloul, F., Lelieveldt, B. P. F., Boudraa, A., and Reiber, J. H. C, “Optimal design of radial basis function neural networks for fuzzy rule extraction in high dimensional data,” Pattern Recognition, 35, 2002, 659–675.
Boyen, X. and Wehenkel, L., “Automatic induction of fuzzy decision trees and its application to power system security assessment,” Fuzzy Sets and Systems, 102, 1999, 3–19.
Carse, B., Fogarty, T. C, and Munro, A., “Evolving fuzzy rule based controllers using genetic algorithms,” Fuzzy Sets and Systems, 80, 1996, 273–293.
Castro, J. L. and Zurita, J. M., “An inductive learning algorithm in fuzzy systems,” Fuzzy Sets and Systems, 89, 1997, 193–203.
Castro, J. L., Castro-Schez, J. J., and Zurita, J. M., “Learning maximal structure rules in fuzzy logic for knowledge acquisition in expert systems,” Fuzzy Sets and Systems, 101, 1999, 331–342.
Castro, J. L., Castro-Schez, J. J., and Zurita, J. M, “Use of a fuzzy machine learning technique in the knowledge acquisition process,” Fuzzy Sets and Systems, 123, 2001, 307–320.
Chakraborty, D. and Pal, N. R., “Integrated feature analysis and fuzzy rule-based system identification in a neuro-fuzzy paradigm,” IEEE Trans. Systems, Man, Cybernetics-Part B: Cybernetics, 31(3), 2001, 391–400.
Chan, K. C, Lin, G. C. I., and Leong, S. S., “A more accurate adaptive fuzzy inference system”, Computers in Industry, 26, 1995, 61–73.
Chen, J.-L. and Chang, J.-Y., “Fuzzy perceptron neural networks for classifiers with numerical data and linguistic rules as inputs,” IEEE Trans. On Fuzzy Systems, 8(6), 2000, 730–745.
Chen, M.Y. and Likens, D. A., “A systematic neuro-fuzzy modeling framework with application to material property prediction,” IEE Trans. Systems, Man, Cybernetics-Part B: Cybernetics, 31(5), 2001, 781–790.
Chen, S.-M. and Yeh, M.-S., “Generating fuzzy rules from relational database systems for estimating null values,” Cybernetics and Systems, 28, 1997, 695–723.
Chi, Z. and Yan, H., “ID3-derived fuzzy rules and optimized defuzzification for handwritten numeral recognition,” IEEE Trans. On Fuzzy Systems, 4(1), 1996, 24–31.
Chiang, I.-J. and Hsu, J. Y., “Fuzzy classification trees for data analysis,” Fuzzy Sets and Systems, 130, 2002, 87–99.
Cho, H.-J., Cho, K.-B., and Wang, B.-H., “Fuzzy-PID hybrid control: automatic rule generation using genetic algorithms”, Fuzzy Sets and Systems, 92, 1997, 305–316.
Cordón, O. and Herrera, F., “A three-stage evolutionary process for learning descriptive and approximate fuzzy-logic-controller knowledge base from examples,” Int. J. of Approximate Reasoning, 17, 1997, 369–407.
Cordón, O. and Herrera, F., “A proposal for improving the accuracy of linguistic modeling”, IEEE Trans. Fuzzy Systems, 8(3), 2000, 335–344.
Cordón, O. and Herrera, F., “Hybridizing genetic algorithms with sharing scheme and evolution strategies for designing approximate fuzzy rule-based systems,” Fuzzy Sets and Systems, 118, 2001, 235–255.
Cordón, O., Herrera, F., and Lozano, M., “On the combination of fuzzy logic and evolutionary computation: A short review and bibliography,” in Fuzzy Evolutionary Computation, W. Pedrycz, Ed., Norwell, MA: Kluwer, 1997, 33–56.
Cordón, O., del Jusus, M. J., and Herrera, F., “Genetic learning of fuzzy rule-based classification systems cooperating with fuzzy reasoning methods,” Int. J. Intelligent Systems, 13, 1998, 1025–1053.
Delgado, M., Gómez-Skarmeta, A. F., and Martin, F., “A fuzzy clustering-based rapid prototyping for fuzzy rule-based modeling”, IEEE Trans. Fuzzy Systems, 5(2), 1997, 223–232.
Delgado, M. and Gonzalez, A., “An inductive learning procedure to identify fuzzy systems”, Fuzzy Sets and Systems, 55, 1993, 121–132.
Dong, M. and Kothari, R., “Look-ahead based fuzzy decision tree induction,” IEEE Trans. On Fuzzy Systems, 9(3), 2001, 461–468.
Dueck, G. and Scheuer, T., “Threshold accepting: A general purpose optimization algorithm appearing superior to simulated annealing,” J. Comput. Phys., 90, 1990, 161–175.
Emami, M. R., Türksen, I. B., and Goldenberg, A. A., “Development of a systematic methodology of fuzzy logic modeling”, IEEE Trans. Fuzzy Systems, 6(3), 1998, 346–361.
Enbutsu, I., Baba, K., and Hara, N., “Fuzzy rule extraction from a multilayereed neural network”, Proc. Of IJCNN’ 91, Seattle, 1991, II-461–465.
Espinosa, J. and Vandewalle, J., “Constructing fuzzy models with linguistic integrity from numerical data-AFRELI algorithm”, IEEE Trans. Fuzzy Systems, 8(5), 2000, 591–600.
Gómez-Skarmeta, A. F. and Jiménez, F., “Fuzzy modeling with hybrid systems”, Fuzzy Sets and Systems, 104, 1999, 199–208.
Guillaume, S., “Designing fuzzy inference systems from data: an interpretability-oriented review”, IEEE Trans. Fuzzy Systems, 9(3), 2001, 426–443.
Heider H. and Drabe, T., “A cascaded genetic algorithm for improving fuzzy system design,” Int. J. Approximate Reasoning, 17, 1997, 351–368.
Herrera, F., Lozano, M. and Verdegay, J. L., “A learning process for fuzzy control rules using genetic algorithms”, Fuzzy Sets and Systems, 100, 1998, 143–158.
Hiraga, I., Furuhashi, T., Uchikawa, Y., and Nakayama, S., “An acquisition of operator’s rules for collision avoidance using fuzzy neural networks,” IEEE Trans. Fuzzy Systems, 3(3), 1995, 280–287.
Hirota, K. and Pedrycz, W., “Directional fuzzy clustering and its application to fuzzy modeling,” Fuzzy Sets and Systems, 80, 1996, 315–326.
Homaifar, A. and McCormick, E., “Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms”, IEEE Trans. Fuzzy Systems, 3(2), 1995, 129–138.
Hong, J. R., “AE1: extension matrix approximate method for general covering problem,” Int. J. Comput. Inform. Sci, 14(6), 1985, 421–437.
Hong, T. P. and Chen, J.-B., “Finding relevant attributes and membership functions,” Fuzzy Sets and Systems, 103, 1999, 389–404.
Hong, T. P. and Chen, J.-B., “Processing individual fuzzy attributes for fuzzy rule induction,” Fuzzy Sets and Systems, 112, 2000, 127–140.
Huang, S. H. and Xing, H., “Extract intelligent and concise fuzzy rules from neural networks,” Fuzzy Sets and Systems, 132, 2002, 233–243.
Ishibuchi, H., Murata, T., and Türksen, I. B., “Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems,” Fuzzy Sets and Systems, 89, 1997, 135–150.
Ishibuchi, H., Nozaki, K., and Tanaka, H., “Distributed representation of fuzzy rules and its application to pattern recognition”, Fuzzy Sets and Systems, 52, 1992, 21–32.
Ishibuchi, H., Nozaki, K., and Tanaka, H., “Efficient fuzzy partition of pattern space for classification problems”, Fuzzy Sets and Systems, 59, 1993, 295–304.
Ishibuchi, H., Nozaki, K., Yamamoto, N. and Tanaka, H., “Selecting fuzzy If-Then rules for classification problems using genetic algorithms,” IEEE Trans. Fuzzy Systems, 3(3), 1995, 260–270.
Ishihashi, H., Shirai, T., Nagasaka, K., and Miyoshi, T., “Neuro-fuzzy ID3: a method of inducing fuzzy decision trees with linear programming for maximizing entropy and an algebraic method for incremental learning,” Fuzzy Sets and Systems, 81, 1996, 157–167.
Janikow, C. Z., “A genetic algorithm method for optimizing fuzzy decision trees,” Information Sciences, 89, 1996, 275–296.
Janikow, C. Z., “Fuzzy decision trees: issues and methods,” IEEE Trans. Systems, Man, and Cybernetics-Part B: Cybernetics, 28(1), 1998, 1–14.
Jeng, B., Heng, Y.-M., and Liang, T. P., “FILM: a fuzzy inductive learning method for automated knowledge acquisition,” Decision Support Systems, 21, 1997, 61–73.
Kang, S.-J., Woo, C.-H., Hwang, H.-S., Woo, K. B., “Evolutionary design of fuzzy rule base for nonlinear system modeling and control,” IEEE Trans. Fuzzy Systems, 8(1), 2000, 37–44.
Kaur, D. and Lin, B., “On the design of neural-fuzzy control system”, Int. J. of Intelligent Systems, 13, 1998, 11–26.
Kbir, M. A., Benkirane, H., Maalmi, K., and Benslimane, R., “Hierarchical fuzzy partition for pattern classification with fuzzy If-Then rules”, Pattern Recognition Letters, 21, 2000, 503–509.
Kilic, K, Sproule, B. A., Türksen, I. B., and Naranjo, C. A., “Fuzzy system modeling in pharmacology: an improved algorithm,” Fuzzy Sets and Systems, 130, 2002, 253–264.
Kim, C. J., “An algorithmic approach for fuzzy inference,” IEEE Trans. Fuzzy Systems, 5(4), 1997, 585–598.
Klawonn, F. and Kruse, R., “Constructing a fuzzy controller from data”, Fuzzy Sets and Systems, 85, 1997, 177–193.
Lekova, A., Mikhailov, L., Boyadjiev, D., and Nabout, A., “Redundant fuzzy rules exclusion by genetic algorithms”, Fuzzy Sets and Systems, 100, 1998, 235–243.
Li, R. P., Mukaidono, M., and Türksen, I. B., “A fuzzy neural network for pattern recognition and feature selection,” Fuzzy Sets and Systems, 130, 2002, 101–108.
Liao, T. W., “A fuzzy c-medians variant for the generation of fuzzy term sets,” J. Intelligent Systems, 17(1), 2001.
Liao, T. W., Celmins, A. K., and Hammell, R. J., II, “A fuzzy c-means variant for the generation of fuzzy term sets,” Fuzzy Sets and Systems, 135, 2003, 241–257.
Lin, C.-T. and Lee, C. S. G., “Neural network-based fuzzy logic control and decision system”, IEEE Trans. Computing, 40(12), 1991, 1320–1336.
Lin, C.-T. and Lu, Y.-C., “A neural fuzzy system with linguistic teaching signals,” IEEE Trans. Fuzzy Systems, 3(2), 1995, 169–189.
Lin, Y. and Cunningham, G. A., II, “A new approach to fuzzy-neural system modeling,” IEEE Trans. Fuzzy Systems, 3, 1995, 190–198.
Lin, Y., Cunningham, G. A., II, and Coggeshall, S. V., “Using fuzzy partitions to create fuzzy systems from input-output data and set the initial weights in a fuzzy neural network”, IEEE Trans. Fuzzy Systems, 5(4), 1997, 614–621.
Mamdani, E. H. and Assilian, S., “An experiment in linguistic synthesis with a fuzzy logic controller”, Int. J. Man-Mach. Stud., 7, 1975, 1–13.
Mikhailov, L., Lekova, A., and Nabout, A., “Method for fuzzy rule extraction from numerical data”, Complex Systems, 1, 1996, 23–33.
Mitra, S., De, R. K., and Pal, S. K., “Knowledge-based fuzzy MLP for classification and rule generation”, IEEE Trans. Neural Networks, 8(6), 1997, 1338–1350.
Mitra, S. and Hayashi, Y., “Neuro-fuzzy rule generation: survey in soft computing framework,” IEEE Trans. Neural Networks, 11(3), 2000, 748–768.
Nauck, D., Klawonn, F., and Kruse, R., Foundations of Neuro-Fuzzy Systems. Chichester, U. K.: Wiley, 1997.
Nauck, D. and Kruse, R., “A neuro-fuzzy method to learn fuzzy classification rules from data,” Fuzzy Sets and Systems, 89, 1997, 277–288.
Nawa, N. E. and Furuhashi, T., “Fuzzy system parameters discovery by bacterial evolutionary algorithm,” IEEE Trans. Fuzzy Systems, 7(5), 1999, 608–616.
Nozaki, K., Ishibuchi, H., and Tanaka, H., “Adaptive fuzzy rule-based classification systems”, IEEE Trans. Fuzzy Systems, 4(3), 1996, 238–250.
Pal, N. R. and Chakraborty, S., “Fuzzy rule extraction from ID3-type decision trees for real data,” IEEE Trans. On Systems, Man, and Cybernetics-Part B: Cybernetics, 31(5), 2001, 745–754.
Pedrycz, W. “Fuzzy set technology in knowledge discovery,” Fuzzy Sets and Systems, 98, 1998, 279–290.
Peña-Reyes, C. A. and Sipper, M., “Fuzzy CoCo: a cooperative-coevolutionary approach to fuzzy modeling”, IEEE Trans. On Fuzzy Systems, 9(5), 2001, 727–737.
Quek, C. and Tung, W. L., “A novel approach to the derivation of fuzzy membership functions using the Falcon-MART architecture”, Pattern Recognition Letters, 22, 2001, 941–958.
Quek, C. and Zhou, R. W., “The POP learning algorithms: reducing work in identifying fuzzy rules,” Neural Networks, 14, 2001, 1431–1445.
Quinlan, J. R., “Induction of decision trees,” Machine Learning, 1(1), 1986, 81–106.
Ravi, V., Reddy, P. J., and Zimmermann, H.-J., “Fuzzy rule base generation for classification and its minimization via modified threshold accepting”, Fuzzy Sets and Systems, 120, 2001, 271–279.
Russo, M., “FuGeNeSys-A fuzzy genetic neural system for fuzzy modeling”, IEEE Trans. Fuzzy Systems, 6(3), 1998, 373–388.
Russo, M., “Genetic fuzzy learning”, IEEE Trans. Evolutionary Computation, 4(3), 2000, 259–273.
Rutkowska, D., “On generating fuzzy rules by an evolutionary approach”, Cybernetics and Systems, 29, 1998, 391–407.
Safavian, S. R. and Landgrebe, D., “A survey of decision tree classifier methodology,” IEEE Trans. Systems Man Cybernet, 21, 1991, 600–674.
Shi, Y., Eberhart, R., and Chen Y., “Implementation of evolutionary fuzzy systems,” IEEE Trans. On Fuzzy Systems, 7(2), 1999, 109–119.
Su, M.-C. and Chang, H.-T., “Application of neural networks incorporated with real-valued genetic algorithms in knowledge acquisition,” Fuzzy Sets and Systems, 112, 2000, 85–97.
Sugeno, M. and Kang, G. T., “Fuzzy modeling and control of multiplayer incinerator”, Fuzzy Sets and Systems, 18, 1986, 329–346.
Sugeno, M. and Yasukawa, T., “A fuzzy-logic-based approach to qualitative modeling”, IEEE Trans. Fuzzy Systems, 1(1), 1993, 7–31.
Surmann, H. and Selenschtschitow, A., “Automatic generation of fuzzy logic rule bases: Examples I,” Proc. Of the NF2002: First Int. ICSC Conf. on Neuro-fuzzy Technologies, Cuba, Jan. 16–19, 2002, 75–81.
Takagi, T. and Sugeno, M., “Fuzzy identification of systems and its application to modeling and control”, IEEE Trans. Syst. Man, Cybernet., 15, 1985, 116–132.
Tarng, Y. S., Yeh, Z. M., and Nian, C. Y., “Genetic synthesis of fuzzy logic controllers in turning”, Fuzzy Sets and Systems, 83, 1996, 301–310.
Tsang, E. C. C., Wang, X. Z., and Yeung, D. S., “Improving learning accuracy of fuzzy decision trees by hybrid neural networks,” IEEE Trans. On Fuzzy Systems, 8(5), 2000, 601–614.
Umanol, M., Okamoto, H., Hatono, I., Tamura, H., Kawachi, F., Umedzu, S., and Kinoshita, J., “Fuzzy decision trees by fuzzy ID3 algorithm and its application to diagnosis systems,” in Proc. IEEE Int. Conf. on Fuzzy Systems, June 26–29, 1994, 2113–2118.
Wang, X. Z., Chen, B., Qian, G., and Ye, F., “On the optimization of fuzzy decision trees,” Fuzzy Sets and Systems, 112, 2000, 117–125.
Wang, X. and Hong, J., “On the handling of fuzziness for continuous-valued attributes in decision tree generation,” Fuzzy Sets and Systems, 99, 1998, 283–290.
Wang, C.-H., Liu, J.-F., Hong, T.-P., and Tseng, S.-S., “A fuzzy inductive learning strategy for modular rules,” Fuzzy Sets and Systems, 103, 1999, 91–105.
Wang, L.-X. and Mendel, J. M., “Generating fuzzy rules by learning from examples”, IEEE Trans. Syst. Man Cybernet., 22(6), 1992, 1414–1427.
Wang, X. Z., Wang, Y. D., Xu, X. F., Ling, W. D., and Yeung, D. S., “A new approach to fuzzy rule generation: fuzzy extension matrix,” Fuzzy Sets and Systems, 123, 2001a, 291–306.
Wang, X. Z., Yeung, D. S., and Tsang, E. C. C., “A comparative study on heuristic algorithms for generating fuzzy decision trees,” IEEE Trans. On Systems, Man, and Cybernetics-Part B: Cybernetics, 31(2), 2001b, 215–226.
Wu, T.-P. and Chen, S.-M., “A new method for constructing membership functions and fuzzy rules from training examples”, IEEE Trans. Syst. Man, Cybernet.-Part B: Cybernetics, 29(1), 1999, 25–40.
Wu, X., “Fuzzy interpretation of discretized intervals,” IEEE Trans. On Fuzzy Systems, 7(6), 1999, 753–759.
Xiong, N. and Litz, L., “Reduction of fuzzy control rules by means of premise learning — method and case study,” Fuzzy Sets and Systems, 132, 2002, 217–231.
Yang, Y., Xu, X., and Zhang, W., “Real-time stable self-learning FNN controller using genetic algorithm,” Fuzzy Sets and Systems, 100, 1998, 173–178.
Yao, J., Dash, M., Tan, S. T., and Liu, H., “Entropy-based fuzzy clustering and fuzzy modeling,” Fuzzy Sets and Systems, 113, 2000, 381–388.
Yeung, D. S., Wang, X. Z., and Tsang, E. C. C., “Learning weighted fuzzy rules from examples with mixed attributes by fuzzy decision trees,” in Proc. IEEE Int. Conf. on Systems, Man, and Cybernetics, Tokyo, Japan, Oct. 12–15, 1999, 349–354.
Yuan, Y. and Shaw, M. J., “Induction of fuzzy decision trees,” Fuzzy Sets and Systems, 69, 1995, 125–139.
Zadeh, L. A., “Outline of a new approach to the analysis of complex systems and decision processes”, IEEE Trans. Syst. Man, Cybernet., SMC-3, 28–44, 1973. Zadeh, L. A., “Fuzzy sets”, Information and Control, 8, 1965, 338–353.
Zhang, J. and Knoll, A., “Designing fuzzy controllers by rapid learning,” Fuzzy Sets and Systems, 101, 1999, 287–301.
Zhang, Y.-Q and Kandel, A., “Compression and expansion of fuzzy rule bases by using crisp-fuzzy neural networks,” Cybernetics and Systems, 29, 1998, 6–34.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Liao, T.W. (2006). Mining Human Interpretable Knowledge with Fuzzy Modeling Methods: An Overview. In: Triantaphyllou, E., Felici, G. (eds) Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques. Massive Computing, vol 6. Springer, Boston, MA . https://doi.org/10.1007/0-387-34296-6_15
Download citation
DOI: https://doi.org/10.1007/0-387-34296-6_15
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-34294-8
Online ISBN: 978-0-387-34296-2
eBook Packages: Computer ScienceComputer Science (R0)