Parallel Cooperation for Large-Scale Multiobjective Optimization on Feature Selection Problems

  • Dragi Kimovski
  • Julio OrtegaEmail author
  • Andrés Ortiz
  • Raúl Baños
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9028)


Recently, the interest on multiobjective optimization problems with a large number of decision variables has grown since many significant real problems, for example on machine learning and pattern recognition, imply to process patterns with a high number of components (features). This paper deals with parallel multiobjective optimization on high-dimensional feature selection problems. Thus, several parallel multiobjective evolutionary alternatives based on the cooperation of subpopulations are proposed and experimentally evaluated by using some synthetic and BCI (Brain-Computer Interface) benchmarks. The results obtained show different improvements achieved in the solution quality and speedups, depending on the parallel alternative and benchmark profile. Some alternatives even provide superlinear speedups with only small reductions in the solution quality.


Cooperative coevolution Feature selection Multiobjective clustering Parallel evolutionary algorithms 



This work has been funded by projects TIN2012-32039 (Spanish “Ministerio de Economía y Competitividad” and FEDER funds) and P11-TIC-7983 (“Junta de Andalucía”).


  1. 1.
    Durillo, J., Nebro, A., Coello Coello, C.A., García-Nieto, J., Luna, F., Alba, E.: A study of multiobjective metaheuristics when solving parameter scalable problems. IEEE Trans. Evol. Comput. 14(4), 618–635 (2010)CrossRefGoogle Scholar
  2. 2.
    Antonio, L.M., Coello Coello, C.A.: Use of cooperative coevolution for solving large scale multiobjective optimization problems. In: IEEE Congress on Ecolutionary Computation, pp. 2758–2765, 20–23 June 2013, Cancún, Mexico, vol. 43, no. 2, pp. 445–463 (2013)Google Scholar
  3. 3.
    Raudys, S.J., Jain, A.K.: Small sample size effects in statistical pattern recognition: recommendations for practitioners. IEEE Trans. Pattern Anal. Mach. Intell. 13(3), 252–264 (1991)CrossRefGoogle Scholar
  4. 4.
    Saeys, Y., Inza, I., Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)CrossRefGoogle Scholar
  5. 5.
    Acir, N., Güzeliş, C.: An application of support vector machine in bioinformatics: automated recognition of epileptiform patterns in EEG using SVM classifier designed by a perturbation method. In: Yakhno, T. (ed.) ADVIS 2004. LNCS, vol. 3261, pp. 462–471. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Sun, Z.: Parallel feature selection based on MapReduce. In: Wong, W.E., Zhu, T. (eds.) Computer Engineering and Networking. LNEE, vol. 277, pp. 299–306. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  7. 7.
    Zao, Z., Zhang, R., Cox, J., Duling, D., Sarle, W.: Massively parallel feature selection: an approach based on variance preservation. Mach. Learn. 92, 195–220 (2013)CrossRefMathSciNetGoogle Scholar
  8. 8.
    de Souza, J.T., Matwin, S., Japkowitz, N.: Parallelizing feature selection. Algoritmica 45, 433–456 (2006)CrossRefzbMATHGoogle Scholar
  9. 9.
    Handl, J., Knowles, J.: Feature selection in unsupervised learning via multi-objective optimization. Int. J. Comput. Intell. Res. 2(3), 217–238 (2006)MathSciNetGoogle Scholar
  10. 10.
    Alba, E.: Parallel evolutionary algorithms can achieve super-linear performance. Inf. Process. Lett. 82(1), 7–13 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Potter, M., De Jong, K.A.: A cooperative coevolutionary approach to function optimization. In: Davidor, Y., Männer, R., Schwefel, H.-P. (eds.) PPSN 1994. LNCS, vol. 866, pp. 249–257. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  12. 12.
    Coello Coello, C.A., Lamont, G.B., Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-objective Problems (Chapter 3), 2nd edn. Springer, New York (2007)Google Scholar
  13. 13.
    Mao, J., Hirasawa, K., Murata, J.: Genetic symbiosis algorithm for multi-objective optimization problem. In: Proceedings of the 2000 IEEE International Workshop on Robot and Human Interactive Communication, pp. 137–142 (2000)Google Scholar
  14. 14.
    Keerativuttitumrong, N., Chaiyaratana, N., Varavithya, V.: Multi-objective co-operative co-evolutionary genetic algorithm. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 288–297. Springer, Heidelberg (2002)Google Scholar
  15. 15.
    Coello Coello, C.A., Sierra, M.R.: A coevolutionary multi-objective evolutionary algorithm. In: IEEE Congress on Evolutionary Computation, vol. 1, pp. 482–489 (2003)Google Scholar
  16. 16.
    Maneeratana, K., Boonlong, K., Chaiyaratana, N.: Multi-objective optimisation by co-operative co-evolution. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 772–781. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  17. 17.
    Iorio, A.W., Li, X.: A cooperative coevolutionary multiobjective algorithm using non-dominated sorting. In: Deb, K., Tari, Z. (eds.) GECCO 2004. LNCS, vol. 3102, pp. 537–548. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Tan, K.C., Yang, Y.J., Goh, C.K.: A distributed cooperative coevolutionary algorithm for multiobjective optimization. IEEE Trans. Evol. Comput. 10(5), 527–549 (2006)CrossRefGoogle Scholar
  19. 19.
    Goh, C.-K., Tan, K.C.: A coevolutionary paradigm for dynamic multi-objective optimization. In: Goh, C.-K., Tan, K.C. (eds.) Evolutionary Multi-objective Optimization in Uncertain Environments. SCI, vol. 186, pp. 153–185. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  20. 20.
    Dorronsoro, B., Danoy, G., Nebro, A.J., Boubry, P.: Achieving super-linear performance in parallel multi-objective evolutionary algorithms by means of cooperative coevolution. Comput. Oper. Res. 40, 1552–1563 (2013)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Kohonen, T.: Self-organizing Maps. Springer, Heidelberg (2001)CrossRefzbMATHGoogle Scholar
  22. 22.
    Kimovski, D., Ortega, J., Ortiz, A., Baños, R.: Feature selection in high-dimensional EEG data by parallel multi-objective optimization. In: Proceedings of the IEEE Cluster, Madrid, pp. 314–322 (2014)Google Scholar
  23. 23.
  24. 24.
    Deb, K., Agrawal, S., Pratab, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithms for multi-objective optimisation: NSGA-II. In: Deb, K., Rudolph, G., Lutton, E., Merelo, J.J., Schoenauer, M., Schwefel, H.-P., Yao, X. (eds.) PPSN 2000. LNCS, vol. 1917, pp. 849–858. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  25. 25.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Academic Press, New York (2009)Google Scholar
  26. 26.
    Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience, New York (1998)zbMATHGoogle Scholar
  27. 27.
    Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Dragi Kimovski
    • 1
  • Julio Ortega
    • 2
    Email author
  • Andrés Ortiz
    • 3
  • Raúl Baños
    • 4
  1. 1.University of Information Science and TechnologyOhridMacedonia
  2. 2.Department of Computer Architecture and TechnologyCITIC, University of GranadaGranadaSpain
  3. 3.Department of Communications EngineeringUniversity of MálagaMálagaSpain
  4. 4.Department of Business Administration and ManagementCatholic University of MurciaMurciaSpain

Personalised recommendations