Skip to main content
SpringerLink
Log in

On fuzzy feature selection in designing fuzzy classifiers for high-dimensional data

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

Generating fuzzy rules for high-dimensional data has been a serious challenge in designing fuzzy rule-based classification systems. For data sets with low dimensions, there are some efficient methods to generate a compact set of short fuzzy rules. However, when the dimensions go up, the number of rules increases exponentially. One solution for lowering the dimensions is feature selection which selects a subset of more effective features. In this regard, a fuzzy feature selection approach is proposed in this paper which tries to choose more relevant features; those which can distinguish the distinct classes well. Our method employs the training patterns in the subspace of some predefined fuzzy sets on each feature and applies their compatibility degrees to evaluate that feature. Since each feature is evaluated individually, this method can be applied efficiently on high-dimensional data. Using the selected features to generate rules in fuzzy rule-based classifiers, this paper also presents a novel criterion to assess each generated rule. This criterion measures the capability of each fuzzy rule in discriminating the positive and negative patterns. To illustrate the scalability of our fuzzy feature selection method beside to the efficiency of generated fuzzy rules, they are applied on some benchmark data sets and the results are compared to some other methods in the literature. The experimental results justify the feasibility of our approach to work with high-dimensional data and its acceptable performance in terms of designing CPU time and classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Alcala-Fdez J, Alcala R, Herrera F (2011a) A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning. IEEE Trans Fuzzy Syst 19(5):857–872

    Article  Google Scholar 

  • Alcala-Fdez J, Fernandez A, Luengo J, Derrac J, Garcia S, Sanchez L, Herrera F (2011b) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Multiple Valued Logic Soft Comput 17(2–3):255–287

    Google Scholar 

  • Almaksour A, Anquetil E (2011) Improving premise structure in evolving Takagi-Sugeno neuro-fuzzy classifiers. Evol Syst 2:25–33

    Article  Google Scholar 

  • Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159(23):3160–3182

    Article  MathSciNet  MATH  Google Scholar 

  • Asuncion A, Newman DJ (2007) UCI machine learning repository. Department of Information and Computer science, University of California, Irvine

    Google Scholar 

  • Bouchachia A, Mittermeir R (2006) Towards incremental fuzzy classifiers. Soft Comput 11(2):193–207

    Article  Google Scholar 

  • Casillas J, Cordon O, Del Jesus MJ, Herrera F (2001) Genetic feature selection in a fuzzy rule-based classification system learning process for high dimensional problems. Inf Sci 136:135–157

    Article  MATH  Google Scholar 

  • Cassillas J, Cordon O, Del Jesus MJ, Herrera F (2001) Genetic feature selection in a fuzzy rule-based classification system learning process for high-dimensional problems. Int J Inf Sci 136:135–157

    MATH  Google Scholar 

  • Chakraborty D, Pal NR (2004) A neuro-fuzzy scheme for simultaneous feature selection and fuzzy rule-based classification. IEEE Trans Neural Netw 15:110–123

    Article  Google Scholar 

  • Chiu S (1994) Fuzzy model identification based on cluster estimation. J Intell Fuzzy Syst 2:276–278

    Google Scholar 

  • Cordon O, Del Jesus MJ, Herrera F, Lozano M (1999) MOGUL: a methodology to obtain genetic fuzzy rule based systems under the iterative rule learning approach. Int J Intell Syst 14(11):1123–1143

    Article  MATH  Google Scholar 

  • Estevez PA, Tesmer M, Perez CA, Zurada JM (2009) Normalized mutual information feature selection. IEEE Trans Neural Netw 20(2):189–201

    Article  Google Scholar 

  • Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7:179–188

    Article  Google Scholar 

  • Gacto MJ, Alcala R, Herrera F (2011) Interpretability of linguistic fuzzy rule-based systems: an overview of interpretability measures. Inf Sci 181(20):4340–4360

    Article  Google Scholar 

  • Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  • Halgamuge S, Glesner M (1994) Neural networks in designing fuzzy systems for real world applications. Fuzzy Sets Syst 65(1):1–12

    Article  Google Scholar 

  • Iglesias JA, Angelov P, Ledezma A, Sanchis A (2010) Evolving classification of agent’s behaviors: a general approach. Evol Syst 1(3):161–172

    Article  Google Scholar 

  • Ishibuchi H, Murata T (1997) Minimizing the fuzzy rule base and maximizing its performance by a multi-objective genetic algorithm. In: Proceedings of 6th FUZZ-IEEE, pp 259–264

  • Ishibuchi H, Nakashima T (2001) Effect of rule weights in fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 9(4):506–515

    Article  Google Scholar 

  • Ishibuchi H, Yamamoto T (2004) Comparison of heuristic criteria for fuzzy rule selection in classification problems. Fuzzy Optim Decis Making 3(2):119–139

    Article  MathSciNet  MATH  Google Scholar 

  • Ishibuchi H, Nakashima T, Morisawa T (1999) Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets Syst 103(2):223–238

    Article  Google Scholar 

  • Kreyszig E (1970) Introductory mathematical statistics. John Wiley, New York

    MATH  Google Scholar 

  • Lee HM, Chen CM, Chen JM, Jou YL (2001) An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans Syst Man Cybern Part B Cybern 31(3):426–432

    Article  Google Scholar 

  • Lughofer E (2011) On-line incremental feature weighting in evolving fuzzy classifiers. Fuzzy Sets Syst 163(1):1–23

    Article  MathSciNet  MATH  Google Scholar 

  • Lughofer E, Buchtala O (2013) reliable all-pairs evolving fuzzy classifiers. IEEE Trans Fuzzy Syst 21(4):625–641

    Article  Google Scholar 

  • Lughofer E, Bouchot J-L, Shaker A (2011) On-line elimination of local redundancies in evolving fuzzy systems. Evol Syst 2(3):165–187

    Article  Google Scholar 

  • Luukka P (2011) Feature selection using fuzzy entropy measures with similarity classifier. Expert Syst Appl 38:4600–4607

    Article  Google Scholar 

  • Mansoori EG, Zolghadri MJ, Katebi SD (2007) A weighting function for improving fuzzy classification systems performance. Fuzzy Sets Syst 158(5):583–591

    Article  MathSciNet  MATH  Google Scholar 

  • Mansoori EG, Zolghadri MJ, Katebi SD (2008) SGERD: a steady-state genetic algorithm for extracting fuzzy classification rules from data. IEEE Trans Fuzzy Syst 16(4):1061–1071

    Article  Google Scholar 

  • Marin-Blazquez JG, Shen Q (2002) From approximative to descriptive fuzzy classifiers. IEEE Trans Fuzzy Syst 10(4):484–497

    Article  Google Scholar 

  • Nauck D, Kruse R (1997) A neuro-fuzzy method to learn fuzzy classification rules from data. Fuzzy Sets Syst 89(3):277–288

    Article  MathSciNet  Google Scholar 

  • Pedrycz W (1994) Why triangular membership functions? Fuzzy Sets Syst 64(1):21–30

    Article  MathSciNet  Google Scholar 

  • Peng H, Long F, Ding C (2005) Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Article  Google Scholar 

  • Pineda-Bautista BB, Carrasco-Ochoa JA, Martinez-Trinidad JF (2011) General framework for class-specific feature selection. Expert Syst Appl 38:10018–10024

    Article  Google Scholar 

  • Rehm F, Klawonn F, Kruse R (2007) Visualization of fuzzy classifiers. Int J Uncertain Fuzziness Knowl Based Syst 15(5):615–624

    Article  Google Scholar 

  • Roubos H, Setnes M (2000) Compact fuzzy models through complexity reduction and evolutionary optimization. Proc Ninth IEEE Int Conf Fuzzy Syst 2:762–767

    Google Scholar 

  • Setnes M, Babuska R, Kaymak U, van Nauta-Lemke HR (1998) Similarity measures in fuzzy rule base simplification. IEEE Trans Syst Man Cybern Part B Cybern 28:376–386

    Article  Google Scholar 

  • Shie JD, Chen SM (2007) Feature subset selection based on fuzzy entropy measures for handling classification problems. Appl Intell 28:69–82

    Article  Google Scholar 

  • Tuv E, Borisov A, Runger G, Torkkola K (2009) Feature selection with ensembles, artificial variables, and redundancy elimination. J Mach Learn Res 10:1341–1366

    MathSciNet  MATH  Google Scholar 

  • Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst 13(2):44–49

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran

    Eghbal G. Mansoori & Khadijeh S. Shafiee

Authors
  1. Eghbal G. Mansoori
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khadijeh S. Shafiee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eghbal G. Mansoori.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mansoori, E.G., Shafiee, K.S. On fuzzy feature selection in designing fuzzy classifiers for high-dimensional data. Evolving Systems 7, 255–265 (2016). https://doi.org/10.1007/s12530-015-9142-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-015-9142-4

Keywords

Search

Navigation