Advertisement

Evolutionary Multiobjective Knowledge Extraction for High-Dimensional Pattern Classification Problems

  • Hisao Ishibuchi
  • Satoshi Namba
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3242)

Abstract

This paper proposes an evolutionary multiobjective optimization (EMO) approach to knowledge extraction from numerical data for high-dimensional pattern classification problems with many continuous attributes. The proposed approach is a three-stage rule extraction method. First each continuous attribute is discretized into several intervals using a class entropy measure. In this stage, multiple partitions with different granularity are specified. Next a prespecified number of candidate rules are generated from numerical data using a heuristic rule evaluation measure in a similar manner to data mining. Then a small number of candidate rules are selected by an EMO algorithm. The EMO algorithm tries to maximize the accuracy of selected rules. At the same time, it tries to minimize their complexity. Our rule selection problem has three objectives: to maximize the number of correctly classified training patterns, to minimize the number of selected rules and to minimize their total rule length. The length of each rule is defined by the number of its antecedent conditions. The main characteristic feature of the proposed EMO approach is that many rule sets with different accuracy and different complexity are simultaneously obtained from its single run. They are tradeoff solutions (i.e., non-dominated rule sets) with respect to the accuracy and the complexity. Through computational experiments, we demonstrate the applicability of the proposed EMO approach to high-dimensional pattern classification problems with many continuous attributes. We also demonstrate some advantages of the proposed EMO approach over single-objective ones.

Keywords

Test Pattern Continuous Attribute Training Pattern Rule Weight Candidate Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., Verkamo, A.I.: Fast Discovery of Association Rules. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 307–328. AAAI Press, Menlo Park (1996)Google Scholar
  2. 2.
    Deb, K.: Multi-Objective Optimization Using Evolutionary Algorithms. John Wiley & Sons, Chichester (2001)MATHGoogle Scholar
  3. 3.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. IEEE Trans. on Evolutionary Computation 6, 182–197 (2002)CrossRefGoogle Scholar
  4. 4.
    Elomaa, T., Rousu, J.: General and Efficient Multisplitting of Numerical Attributes. Machine Learning 36, 201–244 (1999)MATHCrossRefGoogle Scholar
  5. 5.
    Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P.: From Data Mining to Knowledge Discovery: An Overview. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 1–34. AAAI Press, Menlo Park (1996)Google Scholar
  6. 6.
    Gonzalez, A., Perez, R.: SLAVE: A Genetic Learning System Based on an Iterative Approach. IEEE Trans. on Fuzzy Systems 7, 176–191 (1999)CrossRefGoogle Scholar
  7. 7.
    Ishibuchi, H., Murata, T., Turksen, I.B.: Single-Objective and Two-Objective Genetic Algorithms for Selecting Linguistic Rules for Pattern Classification Problems. Fuzzy Sets and Systems 89, 135–150 (1997)CrossRefGoogle Scholar
  8. 8.
    Ishibuchi, H., Nakashima, T., Murata, T.: Three-Objective Genetics-Based Machine Learning for Linguistic Rule Extraction. Information Sciences 136, 109–133 (2001)MATHCrossRefGoogle Scholar
  9. 9.
    Ishibuchi, H., Yamamoto, T.: Fuzzy Rule Selection by Multi-Objective Genetic Local Search Algorithms and Rule Evaluation Measures in Data Mining. Fuzzy Sets and Systems 141, 59–88 (2004)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Lim, T.S., Loh, W.Y., Shih, Y.S.: A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms. Machine Learning 40, 203–228 (2000)MATHCrossRefGoogle Scholar
  11. 11.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo (1993)Google Scholar
  12. 12.
    Weiss, S.M., Kulikowski, C.A.: Computer Systems That Learn. Morgan Kaufmann Publishers, San Mateo (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Hisao Ishibuchi
    • 1
  • Satoshi Namba
    • 1
  1. 1.Department of Industrial EngineeringOsaka Prefecture UniversitySakai, OsakaJapan

Personalised recommendations