Surrogate-Assisted Multi-objective Genetic Algorithms for Fuzzy Rule-Based Classification

  • Harihar Kalia
  • Satchidananda Dehuri
  • Ashish Ghosh
  • Sung-Bae Cho
Article
  • 12 Downloads

Abstract

In this paper, we present a surrogate-assisted multi-objective genetic algorithm to mine a small number of linguistically interpretable fuzzy rules for high-dimensional classification task in the realm of data mining. However, the difficulties like (1) handling of high-dimensional problems by fuzzy rule-based systems (i.e., the exponential increase in the number of fuzzy rules with the number of input variables), (2) the deterioration in the comprehensibility of fuzzy rules when they involve many antecedent conditions, and (3) the optimization of multiple objectives in fuzzy rule-based system may stand as pertinent issues. Hence, to combat with the aforesaid issues, we design the problem as a combinatorial optimization problem with three objectives: to maximize the number of correctly classified training patterns, minimize the number of fuzzy rules, and minimize the total number of antecedent conditions. We optimize these objectives of the fuzzy rule-based system by using a multi-objective genetic algorithm. Further to reduce the number of required evaluations to approximate the Pareto front of computationally expensive multi-objective genetic algorithm for fuzzy rule-based system, a radial basis neural network surrogate model is adapted. This approach searches for non-dominated rule sets with respect to these three objectives. The performance of the surrogate-assisted model is evaluated through a few benchmark datasets obtained from knowledge extraction based on evolutionary learning data repository. The experimental outcome confirm that this model is competitive compared to the non-surrogate-assisted model. However, the performance of the model has drawn a clear edge over rule mining approaches like Decision Table, JRip, OneR, PART, and ZeroR.

Keywords

Classification rule Fuzzy set Genetic algorithm Multi-objective genetic algorithm Surrogate-assisted model 

Notes

Acknowledgements

This work was partly supported by Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (R0124-16-0002), Emotional Intelligence Technology to Infer Human Emotion and Carry on Dialogue Accordingly.

References

  1. 1.
    Witten, I.H., Frank, E., Hall, M.A., Pal, C.: Data Mining: Practical Machine Learning Tools and Techniques, 4th edn. Morgan Kaufmann, Burlington (2016)Google Scholar
  2. 2.
    Weiss, S.M., Kapouleas, I.: An empirical comparison of pattern recognition, neural nets and machine learning classification methods. In: Readings in Machine Learning, pp. 177–183. Morgan Kaufmann Publishers Inc., Burlington (1990)Google Scholar
  3. 3.
    Lim, T.S., Loh, W.Y., Shih, Y.S.: A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms. Mach. Learn. 40(3), 203–228 (2000)CrossRefMATHGoogle Scholar
  4. 4.
    Freitas, A.A.: A survey of evolutionary algorithms for data mining and knowledge discovery. In: Ghosh, A., Tsutsui, S. (eds.) Advances in Evolutionary Computation. Springer, Berlin (2001)Google Scholar
  5. 5.
    Smith, E., Dike, B.A., Stegmann, S.A.: Fitness inheritance in genetic algorithms. In: Proceedings of the 1995 ACM Symposium on Applied Computing, pp. 345–350. ACM Press, New York (1995)Google Scholar
  6. 6.
    Ishibuchi, H., Nakasima, T.: Linguistic rule extraction by genetic-based machine learning. In: Proceedings of Genetic and Evolutionary Computation Conference, pp. 195–202. Morgan Kaufmann (2000)Google Scholar
  7. 7.
    Ishibuchi, H., Murata, T., Turksen, I.B.: Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets Syst. 89(2), 135–150 (1997)CrossRefGoogle Scholar
  8. 8.
    Zimmermann, H.J.: Fuzzy Set Theory and Its Applications. Springer, Berlin (2011)Google Scholar
  9. 9.
    Kramer, O.: Genetic Algorithm Essentials. Studies in Computational Intelligence, vol. 679. Springer, Cham (2017)MATHGoogle Scholar
  10. 10.
    Fernandez, A., Garcia, S., Luengo, J., Bernado-Mansilla, E., Herrera, F.: Genetics-based machine learning for rule induction: state of art, taxonomy and comparative study. IEEE Trans. Evolut. Comput. 14(6), 913–943 (2010)CrossRefGoogle Scholar
  11. 11.
    Herrera, F.: Genetic fuzzy systems: taxonomy, current research trends and prospects. Evolut. Intell. 1(1), 27–46 (2008)CrossRefGoogle Scholar
  12. 12.
    Smith, S.F.: A learning system based on genetic algorithms. Ph.D. dissertation, University of Pittsburg, Pittsburg PA (1980)Google Scholar
  13. 13.
    Ong, Y.S., Nair, P.B., Keane, A.J.: Evolutionary optimization of computationally expensive problems via surrogate modelling. Am. Inst. Aeronaut. Astronaut. J. 41(4), 687–696 (2003)CrossRefGoogle Scholar
  14. 14.
    Jin, Y., Olhofer, M., Sendhoff, B.: A framework for evolutionary optimization with approximate fitness functions. IEEE Trans. Evolut. Comput. 6(5), 481–494 (2002)CrossRefGoogle Scholar
  15. 15.
    Shi, L., Rasheed, K.: A survey of fitness approximation methods applied in evolutionary algorithms. In: Goh, C.K., Tenne, Y. (eds.) Computational Intelligence in Expensive Optimization Problem, pp. 3–28. Springer, Berlin (2010)CrossRefGoogle Scholar
  16. 16.
    Eby, D., Averill, R., Panch, W., Goohman, E .: Evaluation of injection island model GA performance on flywheel design optimization. In: Third Conference on Adaptive Computing in Design and Manufacturing, pp. 121–136. Springer (1998)Google Scholar
  17. 17.
    Sefrioui, M., Periaux, J.: A hierarchical genetic algorithm using multiple models for optimization. In: Parallel Problem Solving From Nature, PPSN VI, pp. 879-888. Springer, Berlin (2000)Google Scholar
  18. 18.
    Abbond, K., Schoenauer, M.: Surrogate Deterministic Mutation. Artificial Evolution, vol. 2310, pp. 103–115. Springer, Berlin (2002)Google Scholar
  19. 19.
    Rasheed, K.: Informed operators: speeding up genetic-algorithm-based design optimization using reduced models. In: Proceedings of Genetic and Evolutionary Computation Conference, pp. 628–635. Morgan Kaufmann, Las Vegas (2000)Google Scholar
  20. 20.
    Papadrakakis, M., Lagaros, N., Tsompanakis, Y.: Optimization of large-scale 3D trusses using evolutionary strategies and neural networks. Int. J. Space Struct. 14(3), 211–223 (1997)CrossRefGoogle Scholar
  21. 21.
    Emmerich, M., Giotis, A., Ozdenir, M., Back, T., Giannakoglou, K.: Meta-model-assisted evolution strategies. In: International Conference on parallel problem solving from nature, pp. 361–370. Springer, Berlin (2002)Google Scholar
  22. 22.
    Jin, Y.: Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement. IEEE Trans. Fuzzy Syst. 8, 212–222 (2000)CrossRefGoogle Scholar
  23. 23.
    Ishibuchi, H., Nakashima, T., Morisawa, T.: Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets Syst. 103, 223–238 (1999)CrossRefGoogle Scholar
  24. 24.
    Ishibuchi, H., Nozaki, K., Yamamoto, N., Tanaka, H.: Selecting fuzzy if-then rules for classification problem using genetic algorithms. IEEE Trans. Fuzzy Syzt. 3(3), 260–270 (1995)CrossRefGoogle Scholar
  25. 25.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. Evolut. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  26. 26.
    Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: improving the performance of the strength of Pareto evolutionary algorithm. Technical Report 103, Computer Engineering and Communication Networks Lab, Swiss Federal Institute of Technology, Zurich (2001)Google Scholar
  27. 27.
    Watanabe, S., Hiroyassu, T., Miki, M.: Multi-objective rectangular packing problem and its applications. In: Proceedings of 2nd International Conference on Multi-criterion Optimization, pp. 565–577 (2003)Google Scholar
  28. 28.
    Ghosh, A., Das, M.K.: Non-dominated rank based sorting genetic algorithms. Fundam. Inf. 83(3), 231–252 (2008)MathSciNetMATHGoogle Scholar
  29. 29.
    Sanchez, L., Otero, J., Couso, I.: Obtaining linguistic fuzzy rule-based regression models from imprecise data with multiobjective genetic algorithms. Soft Comput. 13(5), 467–479 (2009)CrossRefMATHGoogle Scholar
  30. 30.
    Ishibuchi, H., Kuwajima, I., Nojima, Y.: Use of Pareto-optimal and near Pareto-optimal candidate rules in genetic fuzzy rule selection. In: Melin, P., et al. (eds.) Analysis and Design of Intelligent Systems Using Soft Computing Techniques, pp. 387–396. Springer, Berlin (2007)CrossRefGoogle Scholar
  31. 31.
    Ji-lin, C., Yuan-long, H., Zong-yi, X., Li-min, J., Zhong-zhi, T.: A multi-objective genetic-based method for design fuzzy classification systems. Int. J. Comput. Sci. Netw. Secur. 6(8A), 110–117 (2006)Google Scholar
  32. 32.
    Zhang, Y., Wu, X.B., Xing, Z.Y., Hu, W.L.: On generating interpretable and precise fuzzy systems based on Pareto multi-objective cooperative co-evolutionary algorithm. Appl. Soft Comput. 11(1), 1284–1294 (2011)CrossRefGoogle Scholar
  33. 33.
    Nojima, Y., Kaisho, Y., Ishibuchi, H.: Accuracy improvement of genetic fuzzy rule selection with candidate rule addition and membership tuning. In: Proceedings of IEEE International Conference on Fuzzy Systems, Barcelona, pp. 1–8 (2010)Google Scholar
  34. 34.
    Alcalá, R., Nojima, Y., Herrera, F., Ishibuchi, H.: Generating single granularity-based fuzzy classification rules for multi-objective genetic fuzzy rule selection. In: Proceedings of IEEE International Conference on Fuzzy Systems, pp. 1718–1723 (2009)Google Scholar
  35. 35.
    Nojima, Y., Ishibuchi, H.: Incorporation of user preference into multi-objective genetic fuzzy rule selection for pattern classification problems. Artif. Life Robot. 14(3), 418–421 (2009)CrossRefGoogle Scholar
  36. 36.
    Ishibuchi, H., Nakashima, Y., Nojima, Y.: Performance evaluation of evolutionary multiobjective optimization algorithms for multi-objective fuzzy genetics-based machine learning. Soft Comput. 15(12), 2415–2434 (2011)CrossRefGoogle Scholar
  37. 37.
    Alcalá, R., Nojima, Y., Herrera, F., Ishibuchi, H.: Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions. Soft Comput. 15(12), 2303–2318 (2011)CrossRefGoogle Scholar
  38. 38.
    Trawiski, K., Cordon, O., Quirin, A.: A study on the use of multiobjective genetic algorithms for classifier selection in FURIA-based fuzzy multiclassifiers. Int. J. Comput. Intell. Syst. 5(2), 231–253 (2012)CrossRefGoogle Scholar
  39. 39.
    Fazzolari, M., Giglio, B., Alcala, R., Marcelloni, F., Herrera, F.: A study on the application of instance selection techniques in genetic fuzzy rule-based classification systems: accuracy-complexity trade-off. Knowl. Based Syst. 54, 32–41 (2013)CrossRefGoogle Scholar
  40. 40.
    Fazzolari, M., Alcala, R., Herrera, F.: A multi-objective evolutionary method for learning granularities based on fuzzy discretization to improve the accuracy-complexity trade-off of fuzzy rule-based classification systems: D-MOFARC algorithm. Appl. Soft Comput. 24, 470–481 (2014)CrossRefGoogle Scholar
  41. 41.
    Antonelli, M., Ducange, P., Marcelloni, F.: A fast and efficient multi-objective evolutionary learning scheme for fuzzy rule-based classifiers. Inf. Sci. 283(1), 36–54 (2014)MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Gorzaczany, M.B., Filip, R.: A multi-objective genetic optimization for fast, fuzzy rule-based credit classification with balanced accuracy and interpretability. Appl. Soft Comput. 40, 206–220 (2016)CrossRefGoogle Scholar
  43. 43.
    Antonelli, M., Ducange, P., Lazzerini, B., Marcelloni, F.: Multi-objective evolutionary design of granular rule-based classifiers. Granul. Comput. 1(1), 37–58 (2016)CrossRefGoogle Scholar
  44. 44.
    Rudziski, F.: A multi-objective genetic optimization of interpretability-oriented fuzzy rule-based classifiers. Appl. Soft Comput. 38, 118–133 (2016)CrossRefGoogle Scholar
  45. 45.
    Antonelli, M., Ducange, P., Lazzerini, B., Marcelloni, F.: Learning knowledge bases of multi-objective evolutionary fuzzy systems by simultaneously optimizing accuracy, complexity and partition integrity. Soft Comput. 15(12), 2335–2354 (2011)CrossRefGoogle Scholar
  46. 46.
    Gacto, M.J., Alcala, R., Herrera, H.: A multi-objective evolutionary algorithm for tuning fuzzy rule-based systems for preserving interpretability. In: Proceedings of the Joint International Fuzzy Systems Association World Congress and the European Society for Fuzzy Logic and Technology Conference (IFSA/EUSFLAT 2009), pp. 1146–1151 (2009)Google Scholar

Copyright information

© Taiwan Fuzzy Systems Association and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceSeemanta Engineering CollegeMayurbhanjIndia
  2. 2.Department of Information and Communication TechnologyFakir Mohan UniversityBalasoreIndia
  3. 3.Machine Intelligence Unit and Center for Soft Computing ResearchIndian Statistical InstituteKolkataIndia
  4. 4.Department of Computer Science, Yonsei University, Soft Computing LaboratorySeoulSouth Korea

Personalised recommendations