Neural Processing Letters

, Volume 48, Issue 2, pp 863–880 | Cite as

The Influence of Multi-class Feature Selection on the Prediction of Diagnostic Phenotypes

  • Ludwig Lausser
  • Robin Szekely
  • Lyn-Rouven Schirra
  • Hans A. KestlerEmail author


In this work, we evaluate two schemes for incorporating feature selection processes in multi-class classifier systems on high-dimensional data of low cardinality. These schemes operate on the level of the systems’ individual base classifiers and therefore do not perfectly fit in the traditional categories of filter, wrapper and embedded feature selection strategies. They can be seen as two examples of feature selection networks that are only loosely related to the structure of the multi-class classifier system. The architectures are tested for their application in predicting diagnostic phenotypes from gene expression profiles. Their selection stability and the overall generalization ability are evaluated in \(10 \times 10\) cross-validation experiments with support vector machines, random forests and nearest neighbor classifiers on eight publicly available multi-class microarray datasets. Overall the feature selecting multi-class classifier systems were able to outperform their counterparts on at least five of eight datasets.


Multi-class classification Classifier fusion Feature selection High-dimensional data Low cardinality 



The research leading to these results has received funding from the European Community’s Seventh Framework Programme (FP7/2007-2013) under Grant Agreement No. 602783, the German Research Foundation (DFG, SFB 1074 project Z1), and the Federal Ministry of Education and Research (BMBF, Gerontosys II, Forschungskern SyStaR, ID 0315894A and e:Med, SYMBOL-HF, ID 01ZX1407A) all to HAK.

Supplementary material

11063_2017_9706_MOESM1_ESM.pdf (1.3 mb)
Supplementary material 1 (pdf 1326 KB)


  1. 1.
    Alon U, Barkai N, Notterman DA, Gish K, Ybarra S, Mack D, Levine AJ (1999) Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci USA 96(12):6745–6750CrossRefGoogle Scholar
  2. 2.
    Ben-Dor A, Bruhn L, Friedman N, Nachman I, Schummer M, Yakhini Z (2000) Tissue classification with gene expression profiles. J Comput Biol 7(3–4):559–583CrossRefGoogle Scholar
  3. 3.
    Berchtold NC, Cribbs DH, Coleman PD, Rogers J, Head E, Kim R, Beach T, Miller C, Troncoso J, Trojanowski JQ, Zielke HR, Cotman CW (2008) Gene expression changes in the course of normal brain aging are sexually dimorphic. Proc Natl Acad Sci USA 105(40):15,605–15,610CrossRefGoogle Scholar
  4. 4.
    Blum A, Langley P (1997) Selection of relevant features and examples in machine learning. Artif Intell 97(1–2):245–271MathSciNetCrossRefGoogle Scholar
  5. 5.
    Breiman L (2001) Random forests. Mach Learn 45(1):5–32CrossRefGoogle Scholar
  6. 6.
    Chen R, Snyder M (2013) Promise of personalized omics to precision medicine. Wiley Interdiscip Rev Syst Biol Med 5(1):73–82CrossRefGoogle Scholar
  7. 7.
    Cover TM (1965) Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans Electron Comput 14(3):326–334CrossRefGoogle Scholar
  8. 8.
    Dietterich TG, Bariki G (1995) Solving multiclass problems via error-correcting output codes. J Artif Intell Res 2:263–286CrossRefGoogle Scholar
  9. 9.
    Fix E, Hodges JL (1951) Discriminatory analysis: nonparametric discrimination: consistency properties. Tech. Rep. Project 21-49-004, Report Number 4, USAF School of Aviation Medicine, Randolf Field, TexasGoogle Scholar
  10. 10.
    Freund Y, Schapire RE (1995) A decision-theoretic generalization of on-line learning and an application to boosting. In: Vitányi P (ed) Computational learning theory. Lecture Notes in Artificial Intelligence, vol 904. Springer, Berlin, pp 23–37Google Scholar
  11. 11.
    Gobble RM, Qin LX, Brill ER, Angeles CV, Ugras S, O’Connor RB, Moraco NH, DeCarolis PL, Antonescu C, Singer S (2011) Expression profiling of liposarcoma yields a multigene predictor of patient outcome and identifies genes that contribute to liposarcomagenesis. Cancer Res 71(7):2697–2705CrossRefGoogle Scholar
  12. 12.
    Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD, Lander ES (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439):531–537CrossRefGoogle Scholar
  13. 13.
    Gress TM, Kestler HA, Lausser L, Fiedler L, Sipos B, Michalski CW, Werner J, Giese N, Scarpa A, Buchholz M (2011) Differentiation of multiple types of pancreatico-biliary tumors by molecular analysis of clinical specimens. J Mol Med 90(4):457–464CrossRefGoogle Scholar
  14. 14.
    Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182zbMATHGoogle Scholar
  15. 15.
    Haferlach T, Kohlmann A, Wieczorek L, Basso G, Kronnie GT, Béné MC, Vos JD, Hernández JM, Hofmann WK, Mills KI, Gilkes A, Chiaretti S, Shurtleff SA, Kipps TJ, Rassenti LZ, Yeoh AE, Papenhausen PR, Liu WM, Williams PM, Foà R (2010) Clinical utility of microarray-based gene expression profiling in the diagnosis and subclassification of leukemia: report from the international microarray innovations in leukemia study group. J Clin Oncol 28(15):2529–2537CrossRefGoogle Scholar
  16. 16.
    Hastie T, Tibshirani R, Friedman JH (2001) The elements of statistical learning. Springer series in statistics. Springer, New YorkCrossRefGoogle Scholar
  17. 17.
    Huang Y, Suen C (1995) A method of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Trans Pattern Anal Mach Intell 17(1):90–94CrossRefGoogle Scholar
  18. 18.
    Jameson J, Longo D (2015) Precision medicine—personalized, problematic, and promising. N Engl J Med 372(23):2229–2234CrossRefGoogle Scholar
  19. 19.
    Japkowicz N, Shah M (2011) Evaluating learning algorithms: a classification perspective. Cambridge University Press, New YorkCrossRefGoogle Scholar
  20. 20.
    Jones J, Otu H, Spentzos D, Kolia S, Inan M, Beecken WD, Fellbaum C, Gu X, Joseph M, Pantuck AJ, Jonas D, Libermann TA (2005) Gene signatures of progression and metastasis in renal cell cancer. Clin Cancer Res 11(16):5730–5739CrossRefGoogle Scholar
  21. 21.
    Khan J, Wei J, Ringner M, Saal L, Westermann F, Berthold F, Schwab M, Antonesco C, Peterson C, Meltzer P (2001) Classification and diagnostic prediction of cancer using gene expression profiling and artificial neural networks. Nat Med 7(6):673–679CrossRefGoogle Scholar
  22. 22.
    Kimpel MW, Strother WN, McClintick JN, Carr LG, Liang T, Edenberg HJ, McBride WJ (2007) Functional gene expression differences between inbred alcohol-preferring and non-preferring rats in five brain regions. Alcohol 41(2):95–132CrossRefGoogle Scholar
  23. 23.
    Kohavi R, John G (1997) Wrappers for feature subset selection. Artif Intell 97(1–2):273–324CrossRefGoogle Scholar
  24. 24.
    Kuncheva LI (2004) Combining pattern classifiers: methods and algorithms. Wiley, HobokenCrossRefGoogle Scholar
  25. 25.
    Lattke R, Lausser L, Müssel C, Kestler HA (2015) Detecting ordinal class structures. In: Schwenker F, Roli F, Kittler J (eds) Multiple classifier systems, MCS 2015, Lecture notes in computer science, vol 9132. Springer, Cham, pp 100–111CrossRefGoogle Scholar
  26. 26.
    Lausser L, Kestler HA (2010) Robustness analysis of eleven linear classifiers in extremely high-dimensional feature spaces. In: Schwenker F, El Gayar N (eds) Artificial neural networks in pattern recognition. ANNPR 2010, Lecture Notes in Artificial Intelligence, vol 5998. Springer, Berlin, Heidelberg, pp 72–83CrossRefGoogle Scholar
  27. 27.
    Lausser L, Kestler HA (2014) Fold change classifiers for the analysis of gene expression profiles. In: Gaul W, Geyer-Schulz A, Baba Y, Okada A (eds) German–Japanese interchange of data analysis results. Studies in classification, data analysis, and knowledge organization. Springer, Cham, pp 193–202Google Scholar
  28. 28.
    Lausser L, Müssel C, Kestler HA (2013) Measuring and visualizing the stability of biomarker selection techniques. Comput Stat 28(1):51–65MathSciNetCrossRefGoogle Scholar
  29. 29.
    Lorena AC, de Carvalho ACPLF, Gama JMP (2009) A review on the combination of binary classifiers in multiclass problems. Artif Intell Rev 30:19–37CrossRefGoogle Scholar
  30. 30.
    Maire V, Baldeyron C, Richardson M, Tesson B, Salomon AV, Gravier E, Marty-Prouvost B, Koning LD, Rigaill G, Dumont A, Gentien D, Barillot E, Roman-Roman S, Depil S, Cruzalegui F, Pierré A, Tucker GC, Dubois T (2013) TTK/hMPS1 is an attractive therapeutic target for triple-negative breast cancer. PLoS One 8(5):e63712CrossRefGoogle Scholar
  31. 31.
    Müssel C, Lausser L, Maucher M, Kestler HA (2012) Multi-objective parameter selection for classifiers. J Stat Softw 46(5):1–27CrossRefGoogle Scholar
  32. 32.
    Palm G (2016) Neural information processing in cognition: we start to understand the orchestra, but where is the conductor? Front Comput Neurosci 10:1–6CrossRefGoogle Scholar
  33. 33.
    Pfister TD, Reinhold WC, Agama K, Gupta S, Khin SA, Kinders RJ, Parchment RE, Tomaszewski JE, Doroshow JH, Pommier Y (2009) Topoisomerase I levels in the NCI-60 cancer cell line panel determined by validated ELISA and microarray analysis and correlation with indenoisoquinoline sensitivity. Mol Cancer Ther 8(7):1878–1884CrossRefGoogle Scholar
  34. 34.
    Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  35. 35.
    Saeys Y, Iñza I, Larrañaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517CrossRefGoogle Scholar
  36. 36.
    Schirra LR, Lausser L, Kestler HA (2016a) Selection stability as a means of biomarker discovery in classification. In: Wilhelm AFX, Kestler HA (eds) Analysis of large and complex data. Studies in classification, data analysis, and knowledge organization. Springer, Cham, pp 79–89Google Scholar
  37. 37.
    Schirra LR, Schmid F, Kestler HA, Lausser L (2016b) Interpretable classifiers in precision medicine: Feature selection and multi-class categorization. In: Schwenker F, Abbas HM, El Gayar N, Trentin E (eds) Artificial neural networks in pattern recognition, ANNPR 2016, Lecture Notes in Artificial Intelligence, vol 9896. Springer, pp 105–116Google Scholar
  38. 38.
    Skrzypczak M, Goryca K, Rubel T, Paziewska A, Mikula M, Jarosz D, Pachlewski J, Oledzki J, Ostrowski J (2010) Modeling oncogenic signaling in colon tumors by multidirectional analyses of microarray data directed for maximization of analytical reliability. PloS One 5(10):e13091CrossRefGoogle Scholar
  39. 39.
    Taudien S, Lausser L, Giamarellos-Bourboulis EJ, Sponholz C, S F, Felder M, Schirra LR, Schmid F, Gogos C, G S, Petersen BS, Franke A, Lieb W, Huse K, Zipfel PF, Kurzai O, Moepps B, Gierschik P, Bauer M, Scherag A, Kestler HA, Platzer M (2016) Genetic factors of the disease course after sepsis: rare deleterious variants are predictive. EBioMedicine 12:227–238CrossRefGoogle Scholar
  40. 40.
    Vapnik VN (1998) Statistical learning theory. Wiley, New YorkzbMATHGoogle Scholar
  41. 41.
    Webb AR (2002) Statistical pattern recognition, 2nd edn. Wiley, ChichesterCrossRefGoogle Scholar
  42. 42.
    West M, Blanchette C, Dressman H, Huang E, Ishida S, Spang R, Zuzan H, Olson JAJ, Marks JR, Nevins JR (2001) Predicting the clinical status of human breast cancer by using gene expression profiles. Proc Natl Acad Sci USA 98(20):11462–11467CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Institute of Medical Systems BiologyUlm UniversityUlmGermany
  2. 2.Institute of Number Theory and Probability TheoryUlm UniversityUlmGermany
  3. 3.Leibniz Institute on Aging – Fritz Lipmann InstituteJenaGermany

Personalised recommendations