Advertisement

Can the Relevance Index be Used to Evolve Relevant Feature Sets?

  • Laura Sani
  • Riccardo Pecori
  • Emilio Vicari
  • Michele Amoretti
  • Monica Mordonini
  • Stefano Cagnoni
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10784)

Abstract

The Relevance Index (RI) is an information theory-based measure that was originally defined to detect groups of functionally similar neurons, based on their dynamic behavior. More in general, considering the dynamical analysis of a generic complex system, the larger the RI value associated with a subset of variables, the more those variables are strongly correlated with one another and independent from the other variables describing the system status. We describe some early experiments to evaluate whether such an index can be used to extract relevant feature subsets in binary pattern classification problems. In particular, we used a PSO variant to efficiently explore the RI search space, whose size equals the number of possible variable subsets (in this case \(2^{104}\)) and find the most relevant and discriminating feature subsets with respect to pattern representation. We then turned such relevant subsets into a new smaller set of richer features, whose values depend on the values of the binary features they include. The paper reports some exploratory results we obtained in a simple character recognition task, comparing the performance of RI-based feature extraction and selection with other classical feature selection/extraction approaches.

Keywords

Feature selection Feature extraction Information theory Relevance Index 

Notes

Acknowledgments

The authors would like to thank Andrea Roli, Marco Villani, and Roberto Serra for their collaboration, discussions on the topic, and sincere friendship, and Gianluigi Silvestri for implementing K-means PSO in CUDA.

The work of Michele Amoretti was supported by the University of Parma Research Fund - FIL 2016 - Project “NEXTALGO: Efficient Algorithms for Next-Generation Distributed Systems”.

References

  1. 1.
    Villani, M., Filisetti, A., Benedettini, S., Roli, A., Lane, D., Serra, R.: The detection of intermediate level emergent structures and patterns. In: Liò, P., Miglino, O., Nicosia, G., Nolfi, S., Pavone, M. (eds.) Proceedings of ECAL2013, the 12th European Conference on Artificial Life. MIT Press (2013)Google Scholar
  2. 2.
    Tononi, G., McIntosh, A., Russel, D., Edelman, G.: Functional clustering: identifying strongly interactive brain regions in neuroimaging data. Neuroimage 7, 133–149 (1998)CrossRefGoogle Scholar
  3. 3.
    Xue, B., Zhang, M., Browne, W., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 20(4), 606–626 (2016)CrossRefGoogle Scholar
  4. 4.
    Passaro, A., Starita, A.: Particle swarm optimization for multimodal functions: a clustering approach. J. Artif. Evol. Appl. 2008, 8 (2008)Google Scholar
  5. 5.
    Cover, T., Thomas, J.: Element of Information Theory, 2nd edn. Wiley, Hoboken (2006)zbMATHGoogle Scholar
  6. 6.
    Vicari, E., Amoretti, M., Sani, L., Mordonini, M., Pecori, R., Roli, A., Villani, M., Cagnoni, S., Serra, R.: GPU-based parallel search of relevant variable sets in complex systems. In: Rossi, F., Piotto, S., Concilio, S. (eds.) WIVACE 2016. CCIS, vol. 708, pp. 14–25. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-57711-1_2 CrossRefGoogle Scholar
  7. 7.
    Mac Queen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297 (1967)Google Scholar
  8. 8.
    CUDA Toolkit. http://developer.nvidia.com/cuda-toolkit. Accessed 19 Jan 2018
  9. 9.
    Silvestri, G., Sani, L., Amoretti, M., Pecori, R., Vicari, E., Mordonini, M., Cagnoni, S.: Searching relevant variable subsets in complex systems using K-means PSO. In: Roli, A., Slanzi, D., Villani, M. (eds.) Advances in Artificial Life and Evolutionary Computation: 12th Italian Workshop. Springer (2018, in press)Google Scholar
  10. 10.
    Bouckaert, R.R., Frank, E., Hall, M., Kirkby, R., Reutemann, P., Seewald, A., Scuse, D.: WEKA manual for version 3-7-8. University of Waikato, NZ (2013)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Laura Sani
    • 1
  • Riccardo Pecori
    • 1
    • 2
  • Emilio Vicari
    • 3
  • Michele Amoretti
    • 1
  • Monica Mordonini
    • 1
  • Stefano Cagnoni
    • 1
  1. 1.Dipartimento di Ingegneria e ArchitetturaUniversità di ParmaParmaItaly
  2. 2.SMARTEST Research CentreeCampus UniversityNovedrate (CO)Italy
  3. 3.Camlin ItalyParmaItaly

Personalised recommendations