Abstract
We propose a feature selection method for multiclass classification. The proposed method selects features in backward elimination and computes feature ranking scores at each step from analysis of weight vectors of multiple two-class linear Support Vector Machine classifiers from one-versus-one or one-versus-all decomposition of a multi-class classification problem. We evaluated the proposed method on three gene expression datasets for multiclass cancer classification. For comparison, one filtering feature selection method was included in the numerical study. The study demonstrates the effectiveness of the proposed method in selecting a compact set of genes to ensure a good classification accuracy.
Keywords
- Support Vector Machine
- Feature Selection
- Feature Subset
- Feature Selection Method
- Ranking Score
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: Statistical Learning Theory. Wiley Interscience, New York (1998)
Boser, B.E., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Computational Learing Theory, pp. 144–152 (1992)
Friedman, J.H.: Another approach to polychotomous classification. Technical report, Department of Statistics, Stanford University (1996)
Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Machine Learning 46(1-3), 389–422 (2002)
Hsu, C.W., Lin, C.J.: A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 13, 415–425 (2002)
Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. In: Computational Learing Theory, pp. 35–46 (2000)
Weston, J., Watkins, C.: Support vector machines for multiclass pattern recognition. In: Proceedings of the Seventh European Symposium On Artificial Neural Networks (1999)
Duan, K.B., Keerthi, S.S.: Which is the best multiclass SVM method? An empirical study. In: Multiple Classifier Systems, pp. 278–285 (2005)
Duan, K., Keerthi, S.S., Chu, W., Shevade, S.K., Poo, A.N.: Multi-category classification by soft-max combination of binary classifiers. In: Multiple Classifier Systems, pp. 125–134 (2003)
Kreel, U.H.G.: Pairwise classification and support vector machines. In: Scholkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods: Support Vector Learning, pp. 255–268. MIT Press, Cambridge (2002)
Platt, J., Cristianini, N., Shawe-Taylor, J.: Large margin DAGs for multiclass classification (2000)
Hastie, T., Tibshirani, R.: Classification by pairwise coupling. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) Advances in Neural Information Processing Systems, vol. 10, MIT Press, Cambridge (1998)
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. of Artificial Intelligence Research 2, 263–286 (1995)
Rajapakse, J.C., Duan, K.B., Yeo, W.K.: Proteomic cancer classification with mass spectrometry data. American Journal of PharmacoGenomics 5(5), 281–292 (2005)
Duan, K.B., Rajapakse, J.C.: Multiple SVM-RFE for gene selection in cancer classification with expression data. IEEE Trans. Nanobioscience 4(3), 228–234 (2005)
Rakotomamonjy, A.: Variable selection using SVM-based criteria. Journal of Machine Learning Research (Special Issue on Variable Selection) 3, 1357–1370 (2003)
Li, G., Yang, J., Liu, G., Xue, L.: Feature selection for multi-class problems using support vector machines. In: Zhang, C., W. Guesgen, H., Yeap, W.-K. (eds.) PRICAI 2004. LNCS (LNAI), vol. 3157, pp. 292–300. Springer, Heidelberg (2004)
Dudoit, S., Fridlyand, J., Speed, T.P.: Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association 97(457), 77–87 (2002)
Yeoh, E.J., Ross, M.E., Shurtleff, S.A. et al.: Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. Cancer Cell 1, 133–143 (2002)
Armstrong, S.A., Staunton, J.E., Silverman, L.B. et al.: MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nature Genetics 30, 41–47 (2002)
Khan, J., Wei, J.S., Ringn, M., et al.: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nature Medicine 7(6), 673–679 (2001)
Rifkin, R., Klautau, A.: In defense of one-vs-all classification. Journal of Maching Learning Research 5, 101–141 (2004)
Scholkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Duan, KB., Rajapakse, J.C., Nguyen, M.N. (2007). One-Versus-One and One-Versus-All Multiclass SVM-RFE for Gene Selection in Cancer Classification. In: Marchiori, E., Moore, J.H., Rajapakse, J.C. (eds) Evolutionary Computation,Machine Learning and Data Mining in Bioinformatics. EvoBIO 2007. Lecture Notes in Computer Science, vol 4447. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71783-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-71783-6_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71782-9
Online ISBN: 978-3-540-71783-6
eBook Packages: Computer ScienceComputer Science (R0)