On the Use of Variable Complementarity for Feature Selection in Cancer Classification

  • Patrick E. Meyer
  • Gianluca Bontempi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3907)


The paper presents an original filter approach for effective feature selection in classification tasks with a very large number of input variables. The approach is based on the use of a new information theoretic selection criterion: the double input symmetrical relevance (DISR). The rationale of the criterion is that a set of variables can return an information on the output class that is higher than the sum of the informations of each variable taken individually. This property will be made explicit by defining the measure of variable complementarity. A feature selection filter based on the DISR criterion is compared in theoretical and experimental terms to recently proposed information theoretic criteria. Experimental results on a set of eleven microarray classification tasks show that the proposed technique is competitive with existing filter selection methods.


Feature Selection Mutual Information Versus Versus Versus Versus Feature Selection Algorithm Versus Versus Versus Versus Versus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. Journal of Machine Learning Research 3, 1157–1182 (2003)zbMATHCrossRefGoogle Scholar
  2. 2.
    Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artificial Intelligence 97, 273–324 (1997)zbMATHCrossRefGoogle Scholar
  3. 3.
    Blum, A., Langley, P.: Selection of relevant features and examples in machine learning. Artificial Intelligence 97, 245–271 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Provan, G., Singh, M.: Learning bayesian networks using feature selection. In: Fifth International Workshop on Artificial Intelligence and Statistics, pp. 450–456 (1995)Google Scholar
  5. 5.
    Duch, W., Winiarski, T., Biesiada, J., Kachel, A.: Feature selection and ranking filters. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 251–254. Springer, Heidelberg (2003)Google Scholar
  6. 6.
    Bell, D.A., Wang, H.: A formalism for relevance and its application in feature subset selection. Machine Learning 41, 175–195 (2000)zbMATHCrossRefGoogle Scholar
  7. 7.
    Peng, H., Long, F.: An efficient max-dependency algorithm for gene selection. In: 36th Symposium on the Interface: Computational Biology and Bioinformatics (2004)Google Scholar
  8. 8.
    Fleuret, F.: Fast binary feature selection with conditional mutual information. Journal of Machine Learning Research 5, 1531–1555 (2004)MathSciNetGoogle Scholar
  9. 9.
    Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research 5, 1205–1224 (2004)MathSciNetGoogle Scholar
  10. 10.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. John Wiley, New York (1990)Google Scholar
  11. 11.
    Yang, H., Moody, J.: Feature selection based on joint mutual information. In: Advances in Intelligent Data Analysis (AIDA), Computational Intelligence Methods and Applications (CIMA), Rochester New York, ICSC (1999)Google Scholar
  12. 12.
    Kojadinovic, I.: Relevance measures for subset variable selection in regression problems based on k-additive mutual information. Computational Statistics and Data Analysis 49, 1205–1227 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Meyer, P.: Information theoretic filters for feature selection. Technical report, Universite Libre de Bruxelles (548) (2005)Google Scholar
  14. 14.
  15. 15.
    Scott, D.W.: Multivariate Density Estimation. Wiley, Chichester (1992)zbMATHCrossRefGoogle Scholar
  16. 16.

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Patrick E. Meyer
    • 1
  • Gianluca Bontempi
    • 1
  1. 1.Université Libre de BruxellesBruxellesBelgique

Personalised recommendations