Abstract
The feature selection problem can be formulated as a multi-objective optimization (MOO) problem, as it involves the minimization of the feature subset cardinality and the misclassification error. In this chapter, a comparison of MOO algorithms applied to feature selection is presented. The used MOO methods are: Nondominated Sorting Genetic Algorithm II (NSGA-II), Archived Multi Objective Simulated Annealing (AMOSA), and Direct Multi Search (DMS). To test the feature subset solutions, Takagi- Sugeno fuzzy models are used as classifiers. To solve the feature selection problem, AMOSA was adapted to deal with discrete optimization. The multi-objective methods are applied to four benchmark datasets used in the literature and the obtained results are compared and discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Direct multisearch (dms) for multi-objective optimization (2012), http://www.mat.uc.pt/dms/
Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/mlearn/MLRepository.html
Bandyopadhyay, S., Saha, S., Maulik, U., Deb, K.: A simulated annealing-based multi-objective optimization algorithm: AMOSA. IEEE Transactions on Evolutionary Computation 12(3), 269–283 (2008)
van den Berg, J., Kaymak, U., van den Bergh, W.M.: Fuzzy classification using probability-based rule weighting. In: Proc. IEEE Int. Conf. on Fuzzy Systems, vol. 2, pp. 991–996. IEEE Press, Piscataway (2002)
Bhatia, S., Prakash, P., Pillai, G.: SVM based decision support system for heart disease classification with integer-coded genetic algorithm to select critical features. In: Proc. World Congress on Engineering and Computer Science, San Francisco (2008)
Coello, C.C., Voldhuizen, D., Lament, G. (eds.): Evolutionary Algorithms for Solving Multi Objective Problems. Kluwer Academic, New York (2002)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to derivative-free optimization. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2009)
Cordon, O., Herrera, F., Jesus, M., Magdalena, L., Sanchez, A., Villar, P.: A multiobjective genetic algorithm for feature selection and data base learning in fuzzy-rule based classification systems. In: Proc. 9th Int. Conf. on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU 2002), Annecy, France, pp. 823–830 (2002)
Custódio, A.L., Madeira, J.F.A., Vaz, A.I.F., Vicente, L.N.: Direct multisearch for multiobjective optimization. SIAM Journal on Optimization, 10–18 (2011)
Deb, K. (ed.): Multi Objective Optimization using Evolutionary Algorithms. J. Wiley & Sons, Chichester (2004)
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. an Evolutionary Computation 6(2), 10–18 (2002)
Ekbal, A., Saha, S., Garbe, C.: Feature selection using multi-objective optimization for named entity recognition. In: Proc. IEEE Int. Conf. on Pattern Recognition, pp. 1937–1940 (2010)
Emmanouilidis, C. (ed.): Evolutionary Multi-Objective Feature Selection and ROC Analysis with Application to Industrial Machinery Fault Diagnosis. Evolutionary Methods for Design, Optimization and Control. J. Wiley & Sons, Chichester (2002)
Emmanouilidis, C., Hunter, A., MacIntyre, J.: A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In: Proc. Congress on Evolutionary Computation (CEC 2000), San Diego, pp. 823–830 (2000)
Emmanouilidis, C., Hunter, A., MacIntyre, J., Cox, C.: A multi-objective genetic algorithm approach to feature selection in neural and fuzzy modeling. Evolutionary Optimization 3(1), 1–26 (2001)
Gustafson, D., Kessel, W.: Fuzzy identification of systems and its application to modeling and control. IEEE Transactions on Systems, Man and Cybernetics 15(1), 116–132 (1985)
Hamdani, T.M., Won, J.-M., Alimi, A.M., Karray, F.: Multi-objective Feature Selection with NSGA II. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds.) ICANNGA 2007. LNCS, vol. 4431, pp. 240–247. Springer, Heidelberg (2007)
Huang, B., Buckley, B., Kechadi, T.: Multi-objective feature selection by using NSGA-II for customer churn prediction in telecommunications. Expert Systems with Applications 37(5), 3638–3646 (2010)
Lac, H., Stacey, D.: Feature subset selection via multi-objective genetic algorithm. In: Proc. IEEE Int. Joint Conf. on Neural Networks, pp. 1349–1354. IEEE Press, Piscataway (2005)
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Berlin (1999)
Miettinen, K.: Nonlinear Multi-objective Optimization. Kluwer, New York (1999)
Nieto, J., Alba, E., Jourdan, L., Talbi, E.: Sensitivity and specificity based multiobjective approach for feature selection: Application to cancer diagnosis. Information Processing Letters 109, 887–896 (2009)
Oliveira, L., Sabourin, R., Bortolozzi, F., Suen, C.: Feature selection using multi-objective genetic algorithms for handwritten digit recognition. In: Proc. 16th Int. Conf. on Pattern Recognation (ICPR 2002), pp. 568–571. IEEE Press, Piscataway (2002)
Saha, S., Ekbal, A., Uryupina, O., Poesio, M.: Single and multi-objective optimization for feature selection in anaphora resolution. In: Proc. 5th Int. Joint Conf. on Natural Language Processing, pp. 93–101 (2011)
Sousa, J.M.C., Kaymak, U.: Fuzzy Decision Making in Modeling and Control. World Scientific and Imperial College, Singapore and UK (2002)
Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary Computation 2(3), 221–248 (1994)
Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modelling and control. IEEE Trans. on Systems, Man and Cybernetics 15(1), 116–132 (1985)
Unler, A., Murat, A.: A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206, 528–539 (2010)
Vieira, S.M., Sousa, J.M.C., Runkler, T.A.: Two cooperative ant colonies for feature selection using fuzzy models. Expert Systems with Applications 37(4), 2714–2723 (2010)
Wang, C., Huang, Y.: Evolutionary-based feature selection approaches with new criteria for data mining: A case study of credit approval data. Expert Systems with Applications 36, 5900–5908 (2009)
Wang, X., Bandyopadhyay, S., Xuan, Z., Zhao, X., Zhang, M., Zhang, X.: Prediction of transcription start sites based on feature selection using AMOSA. In: Computational Systems Bioinformatics Conference, vol. 6, pp. 183–193 (2007)
Waqas, K., Baig, R., Ali, S.: Feature subset selection using multi-objective genetic algorithms. In: Proc. 13th IEEE Multitopic Conference, pp. 1–6 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag GmbH Berlin Heidelberg
About this chapter
Cite this chapter
Türkşen, Ö., Vieira, S.M., Madeira, J.F.A., Apaydin, A., Sousa, J.M.C. (2013). Comparison of Multi-objective Algorithms Applied to Feature Selection. In: Borgelt, C., Gil, M., Sousa, J., Verleysen, M. (eds) Towards Advanced Data Analysis by Combining Soft Computing and Statistics. Studies in Fuzziness and Soft Computing, vol 285. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30278-7_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-30278-7_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30277-0
Online ISBN: 978-3-642-30278-7
eBook Packages: EngineeringEngineering (R0)