Skip to main content
SpringerLink
Log in

Efficient feature selection methods using PSO with fuzzy rough set as fitness function

  • Application of soft computing
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Feature selection (FS) is defined as the process of selecting a criterion function for evaluating a feature subset and a search strategy for finding the best feature subset from a large number of feature subsets. Many strategies have been established to date, most of which are based on statistical theory; nevertheless, research is currently ongoing to find better solutions in terms of optimality and computing ease. In FS, where most other methods require supplemental knowledge, the rough set ideology of using only the available data and no further information offers several advantages. The key drawback is that the rough set-based feature selection does not function well on data that are real-valued or noisy (continuous datasets). To overcome this problem, we propose three algorithms based on particle swarm optimization with fuzzy rough fitness function for getting optimal feature subset from a feature set with a large number of features (i.e. continuous or discrete datasets). The suggested algorithms are compared against two classical feature selection methods, as well as three PSO and rough set-based feature selection approaches. Seven commonly used discrete datasets with a modest number of features and eight continuous datasets with a large number of features were utilized in the experiments. Two classification techniques (decision tree or DT and Naive Bays or NB) are used to evaluate the classification performance of selected feature subsets. The results show that using the proposed techniques, a small feature subset may be automatically selected with better classification accuracy than utilizing all features. In terms of classification accuracy and amount of features, the suggested algorithms beat the two traditional and three PSO and rough sets-based feature selection approaches. There is also a significant reduction in the number of features with higher weight on the rapid reduct of the fitness function, as well as improved classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Al-Ani A (2005) Feature subset selection using ant colony optimization. Int J Comput Intell 2:53–58

    Google Scholar 

  • Almuallim H, Dietterich TG (1994) Learning Boolean concepts in the presence of many irrelevant features. Art Intell 69(1–2):279

    Article  MathSciNet  Google Scholar 

  • Bache K, Lichman M (2013) UCI machine learning repository

  • Bae C, Yeh WC, Chung YY, Liu SL (2010) Feature selection with intelligent dynamic swarm and rough set. Exp Syst Appl 37(10):7026

    Article  Google Scholar 

  • Barani F, Mirhosseini M, Nezamabadi-pour H (2017) Application of binary quantum-inspired gravitational search algorithm in feature subset selection. Appl Intell pp 1–15

  • Beaubouef T, Petry FE, Arora G (1999) Rough fuzzy hybridization: a new trend in decision-making. Springer, Singapore pp 200–214

  • Cervante L, Xue Shang BL, Zhang M (2013) In European conference on evolutionary computation in combinatorial optimization. Springer, pp 25–36

  • Cervante L, Xue B, Shang L, Zhang M (2012) Australasian joint conference on artificial intelligence. Springer, pp 313–325

  • Chakraborty B (2002) In: IEEE International symposium on industrial electronics (ISIE’02), vol 1, pp 315–319

  • Chakraborty B (2008) In: 3rd International conference on intelligent system and knowledge engineering. ISKE 2008, vol 1 (IEEE, 2008), pp 1038–1042

  • Chen Y, Miao D, Wang R (2010) A rough set approach to feature selection based on ant colony optimization. Pattern Recogn Lett 31(3):226

    Article  Google Scholar 

  • Clerc M (2012) https://hal.archives-ouvertes.fr/hal-00764996

  • Cornelis C, Jensen R, Hurtado G, Śle D et al (2010) Attribute selection with fuzzy decision reducts. Inf Sci 180(2):209

    Article  MathSciNet  Google Scholar 

  • Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1(3):131

    Article  Google Scholar 

  • Dubois D, Prade H (1992) Putting rough sets and fuzzy sets together. Intelligent Decision Support. Springer, pp 203–232

  • Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157

    MATH  Google Scholar 

  • Hall MA (1999) Correlation-based feature selection for machine learning. Ph.D. thesis, The University of Waikato

  • Höhle U (1988) Fuzzy sets and systems. Math Fuzzin 27(1):31

    Google Scholar 

  • Huang CL, Dun JF (2008) A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl Soft Comput 8(4):1381

    Article  Google Scholar 

  • Huda RK, Banka H (2020) New efficient initialization and updating mechanisms in PSO for feature selection and classification. Neural Comput Appl, pp 1–12

  • Huda RK, Banka H (2019) Efficient feature selection and classification algorithm based on PSO and rough sets. Neural Comput Appl 31(8):4287

    Article  Google Scholar 

  • Jensen R (2006) Swarm intelligence in data mining. Springer, pp 45–73

  • Jensen R, Cornelis C (2010)IEEE international conference on fuzzy systems (FUZZ-IEEE), 2010, pp 1–7

  • Jeong YW, Park JB, Jang SH, Lee KY (2010) A new quantum-inspired binary PSO: application to unit commitment problems for power systems. IEEE Trans Power Syst 25(3):1486

    Article  Google Scholar 

  • Ji B, Yuan X, Chen Z, Tian H (2014) Improved gravitational search algorithm for unit commitment considering uncertainty of wind power. Energy 67:52

    Article  Google Scholar 

  • Kohavi R, John GH (1997) Wrappers for feature subset selection. Art Intell 97(1):273

    Article  Google Scholar 

  • Kudo M, Sklansky J (2000) Comparison of algorithms that select features for pattern classifiers. Pattern Recogn 33(1):25–44

    Article  Google Scholar 

  • Liu H, Setiono R (1998) Incremental feature selection. Appl Intell 9(3):217

    Article  Google Scholar 

  • Liu Y, Wang G, Chen H, Dong H, Zhu X, Wang S (2011) An improved particle swarm optimization for feature selection. J Bionic Eng 8(2):191–200

    Article  Google Scholar 

  • Marill T, Green D (1963) On the effectiveness of receptors in recognition systems. IEEE Trans Inf Theory 9(1):11

    Article  Google Scholar 

  • Muni DP, Pal NR, Das J (2006) Genetic programming for simultaneous feature selection and classifier design. IEEE Trans Syst Man Cybern B Cybern 36(1):106

    Article  Google Scholar 

  • Neshatian K, Zhang M(2009) Proceedings of the 11th annual conference on genetic and evolutionary computation. ACM, pp 1027–1034

  • Pawlak Z (2012) Rough sets: theoretical aspects of reasoning about data, vol 9, Springer Science & Business Media

  • Pawlak Z (1982) Rough sets. Int J Comput Inf Sci 11(5):341

    Article  Google Scholar 

  • Pawlak Z, et al (1991) Dordrecht & Boston: Kluwer Academic Publishers

  • Pawlak Z, Skowron A (2007) Rudiments of rough sets. Inf Sci 177(1):3

    Article  MathSciNet  Google Scholar 

  • Pedrycz W (1999) Rough fuzzy hybridization. A new trend in decision-making, pp 179–199

  • Rodrigues D, Pereira LA, Almeida T, Papa JP, Souza A, Ramos CC, Yang XS (2013) 2013 IEEE international symposium on circuits and systems (ISCAS) (IEEE), pp 465–468

  • Shen Q (2005) Combining rough and fuzzy sets for feature selection. In UK workshop on computational intelligence, p 12

  • Shen Q, Chouchoulas A (2000) A modular approach to generating fuzzy rules with reduced attributes for the monitoring of complex systems. Eng Appl Art Intell 13(3):263–278

    Article  Google Scholar 

  • Shi Y, Eberhart R (1998) In: IEEE international conference on evolutionary computation proceedings world congress on computational intelligence (IEEE), pp 69–73

  • Stearns SD (1976) In: Proceedings of the 3rd international joint conference on pattern recognition, pp 71–75

  • Stefanowski J, Tsoukiàs A (2001) Rough sets and current trends in computing. Springer, pp 212–219

  • Stein G, Chen B, Wu AS, Hua KA (2005) In: Proceedings of the 43rd annual Southeast regional conference-vol 2. ACM, pp 136–141

  • Swiniarski RW, Skowron A (2003) Rough set methods in feature selection and recognition. Pattern Recogn Lett 24(6):833

    Article  Google Scholar 

  • Thiele H (1998) HT014601767

  • Tsang EC, Chen D, Yeung DS, Wang XZ, Lee JW (2008) Attributes reduction using fuzzy rough sets. IEEE Trans Fuzzy Syst 16(5):1130

    Article  Google Scholar 

  • Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. Eur J Oper Res 206(3):528

    Article  Google Scholar 

  • Vanderpooten D (1997) Similarity relation as a basis for rough approximations. Adv Mach Intell Soft Comput 4:17

    Google Scholar 

  • Wang X, Yang J, Teng X, Xia W, Jensen R (2007) Feature selection based on rough sets and particle swarm optimization. Pattern Recogn Lett 28(4):459

    Article  Google Scholar 

  • Whitney AW (1971) A direct method of nonparametric measurement selection. IEEE Trans Comput 100(9):1100

    Article  Google Scholar 

  • Whitney AW (1971) A direct method of nonparametric measurement selection. Comput IEEE Trans 100(9):1100

    Article  Google Scholar 

  • Witten IH, Frank E, Hall MA, Pal CJ (2016) Data mining: practical machine learning tools and techniques. Morgan Kaufmann

  • Wygralak M (1989) Rough sets and fuzzy sets-some remarks on interrelations. Fuzzy Sets Syst 29(2):241

    Article  MathSciNet  Google Scholar 

  • Xue B, Cervante L, Shang L, Browne WN, Zhang M (2014) Binary PSO and rough set theory for feature selection: a multi-objective filter based approach. Int J Comput Intell Appl 13(02):1450009

    Article  Google Scholar 

  • Yang XS, Deb S (2009) World Congress on Natural Biol Inspired Comput 2009. NaBIC 2009 (IEEE) pp 210–214

  • Yao Y (1998) A comparative study of fuzzy sets and rough sets. Inf Sci 109(1–4):227–242

    Article  MathSciNet  Google Scholar 

  • Yao Y, Zhao Y (2008) Attribute reduction in decision-theoretic rough set models. Inf Sci 178(17):3356–3373

    Article  MathSciNet  Google Scholar 

  • Zadeh LA (1965) Fuzzy sets. Inf. Control 8(3):338

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Technology (Indian School of Mines), Dhanbad, India

    Ramesh Kumar Huda & Haider Banka

Authors
  1. Ramesh Kumar Huda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haider Banka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramesh Kumar Huda.

Ethics declarations

Conflict of interest

We have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huda, R.K., Banka, H. Efficient feature selection methods using PSO with fuzzy rough set as fitness function. Soft Comput 26, 2501–2521 (2022). https://doi.org/10.1007/s00500-021-06393-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-06393-x

Keywords

Search

Navigation