An Empirical Comparison of Feature Reduction Methods in the Context of Microarray Data Classification

  • Hans A. Kestler
  • Christoph Müssel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4087)


The differentiation between cancerous and benign processes in the body often poses a difficult diagnostic problem in the clinical setting while being of major importance for the treatment of patients. Measuring the expression of a large number of genes with DNA microarrays may serve this purpose. While the expression level of several thousands of genes can be measured in a single experiment, only a few dozens of experiments are normally carried out, leading to data sets of very high dimensionality and low cardinality. In this situation, feature reduction techniques capable of reducing the dimensionality of data are essential for building predictive tools based on classification.

Methods and Data: We compare the popular feature selection and classification method PAM (Tibshirani et al.) to several other methods. Feature reduction and feature ranking methods, such as Random Projection, Random Feature Selection, Area under the ROC curve and PCA are applied. We employ these together with the classification component of PAM, Linear Discriminant Analysis (LDA), a Nearest Prototype (NP) classifier and linear support vector machines (SVMs). We apply these methods to three publicly available linearly separable gene expression data sets of varying cardinality and dimensionality.

Results and Conclusions: In our experiments with the gene expression data we could not discover a clearly superior algorithm, instead most surprisingly we found that feature reduction using random projections or selections performed often equally well.


Principal Component Analysis Linear Discriminant Analysis Feature Reduction Random Projection Empirical Comparison 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Tibshirani, R., Hastie, T., Narasimhan, B., Chu, G.: Diagnosis of multiple cancer types by shrunken centroids of gene expression. PNAS 99(10), 6567–6572 (2002)CrossRefGoogle Scholar
  2. 2.
    Tibshirani, R., Hastie, T., Narasimhan, B., Chu, G.: Class Prediction by Nearest Shrunken Centroids, with Applications to DNA Microarrays. Statistical Science 18(1), 104–117 (2003)MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Vempala, S.: The Random Projection Method. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 65 (2004)Google Scholar
  4. 4.
    Duda, R.O., Hart, P., Storck, D.: Pattern classification. Wiley, Chichester (2001)MATHGoogle Scholar
  5. 5.
    Webb, A.: Statistical Pattern Recognition. Wiley, Chichester (2002)MATHCrossRefGoogle Scholar
  6. 6.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)Google Scholar
  7. 7.
    Tibshirani, R., Hastie, T., Narasimhan, B., Chu, G.: PAMR package version 1.27 (2005),
  8. 8.
    Johnson, W., Lindenstrauss, J.: Extensions of Lipshitz mapping into Hilbert space. Contemporary Mathematics 26, 189–206 (1984)MATHMathSciNetGoogle Scholar
  9. 9.
    Therrien, C.: Decision estimation and classification. Wiley, Chichester (1989)MATHGoogle Scholar
  10. 10.
    Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Haussler, D. (ed.) Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152. ACM Press, New York (1992)CrossRefGoogle Scholar
  11. 11.
    Golub, T., Slonim, D., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J., Coller, H., Loh, M., Downing, J., Caligiuri, M., Bloomfield, C., Lander, E.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–536 (1999)CrossRefGoogle Scholar
  12. 12.
    Dudoit, S., Fridlyand, J., Speed, T.: Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association 97(457), 77–87 (2002)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Khan, J., Wei, J., Ringner, M., Saal, L., Ladanyi, M., Westermann, F., Berthold, F., Schwab, M., Antonescu, C., Peterson, C., Meltzer, P.: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nature Medicine 7(6), 673–679 (2001)CrossRefGoogle Scholar
  14. 14.
    Buchholz, M., Kestler, H., Bauer, A., Bock, W., Rau, B., Leder, G., Kratzer, W., Bommer, M., Scarpa, A., Schilling, M., Adler, G., Hoheisel, J., Gress, T.: Specialized DNA arrays for the differentiation of pancreatic tumors. Clin. Cancer Res. 11(22), 8048–8054 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hans A. Kestler
    • 1
    • 2
  • Christoph Müssel
    • 2
  1. 1.Department of Neural Information ProcessingUniversity of UlmUlmGermany
  2. 2.Department of Internal Medicine IUniversity Hospital UlmUlmGermany

Personalised recommendations