An EA Multi-model Selection for SVM Multiclass Schemes

  • G. Lebrun
  • O. Lezoray
  • C. Charrier
  • H. Cardot
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4507)


Multiclass problems with binary SVM classifiers are commonly treated as a decomposition in several binary sub-problems. An open question is how to properly tune all these sub-problems (SVM hyperparameters) in order to have the lowest error rate for a SVM multiclass scheme based on decomposition. In this paper, we propose a new approach to optimize the generalization capacity of such SVM multiclass schemes. This approach consists in a global selection of hyperparameters for sub-problems all together and it is denoted as multi-model selection. A multi-model selection can outperform the classical individual model selection used until now in the literature. An evolutionary algorithm (EA) is proposed to perform multi-model selection. Experimentations with our EA method show the benefits of our approach over the classical one.


Support Vector Machine Evolutionary Algorithm Combination Scheme Binary Problem Model Selection Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  2. 2.
    Rifkin, R., Klautau, A.: In defense of one-vs-all classification. JMLR 5, 101–141 (2004)MathSciNetGoogle Scholar
  3. 3.
    Price, D., Knerr, S., Personnaz, L., Dreyfus, G.: Pairwise neural network classifiers with probabilistic outputs. In: NIPS, pp. 1109–1116 (1994)Google Scholar
  4. 4.
    Hastie, T., Tibshirani, R.: Classification by pairwise coupling. In: NIPS, pp. 507–513 (1997)Google Scholar
  5. 5.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res (JAIR) 2, 263–286 (1995)zbMATHGoogle Scholar
  6. 6.
    Moreira, M., Mayoraz, E.: Improved pairwise coupling classification with correcting classifiers. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 160–171. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  7. 7.
    Quost, B., Denoeux, T., Masson, M.: Pairwise classifier combination in the framework of belief functions. In: Fusion (2005)Google Scholar
  8. 8.
    Quost, B., Denoeux, T., Masson, M.: One-against-all classifier combination in the framework of belief functions. In: IPMU, vol. 1, pp. 356–363 (2006)Google Scholar
  9. 9.
    Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE Transactions in Neural Networks 13, 415–425 (2002)CrossRefGoogle Scholar
  10. 10.
    Duan, K.-B., Keerthi, S.S.: Which Is the Best Multiclass SVM Method? An Empirical Study. In: Oza, N.C., Polikar, R., Kittler, J., Roli, F. (eds.) MCS 2005. LNCS, vol. 3541, pp. 278–285. Springer, Heidelberg (2005)Google Scholar
  11. 11.
    LeBrun, G., Charrier, C., Lezoray, O., Meurie, C., Cardot, H.: Fast Pixel Classification by SVM Using Vector Quantization, Tabu Search and Hybrid Color Space. In: Gagalowicz, A., Philips, W. (eds.) CAIP 2005. LNCS, vol. 3691, pp. 685–692. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    LeBrun, G., Lezoray, O., Charrier, C., Cardot, H.: A New Model Selection Method for SVM. In: Corchado, E.S., Yin, H., Botti, V., Fyfe, C. (eds.) IDEAL 2006. LNCS, vol. 4224, pp. 99–107. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Lebrun, G.: Model selection methods for SVM (Support Vector Machines). Application in image analysis. PhD thesis, University of Caen (2006)Google Scholar
  14. 14.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), Sofware Available at
  15. 15.
    Platt, J.: Fast training of SVMs using sequential minimal optimization. In: Advances in kernel methods-support vector learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  16. 16.
    Platt, J.: Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In: Smola, A.J., Bartlett, P., Schoelkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers, pp. 61–74 (1999)Google Scholar
  17. 17.
    Wu, T.F., Lin, C.J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 5, 975–1005 (2004)MathSciNetGoogle Scholar
  18. 18.
    Rechenberg, I.: Cybernetic Solution Path of an Experimental Problem. Royal Aircraft Establishment Library Translation (1965)Google Scholar
  19. 19.
    Blake, C., Merz, C.: Uci repository of machine learning databases. In: Advances in kernel methods, support vector learning (1998)Google Scholar
  20. 20.
    LeBrun, G., Lezoray, O., Charrier, C., Cardot, H.: Speed-Up LOO-CV with SVM Classifier. In: Corchado, E.S., Yin, H., Botti, V., Fyfe, C. (eds.) IDEAL 2006. LNCS, vol. 4224, pp. 108–115. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • G. Lebrun
    • 1
  • O. Lezoray
    • 1
  • C. Charrier
    • 1
  • H. Cardot
    • 2
  1. 1.LUSAC EA 2607, Vision and Image Analysis Team, IUT SRC, 120 Rue de l’exode, Saint-Lô, F-50000France
  2. 2.Laboratoire d’Informatique (EA 2101), Université François-Rabelais de Tours, 64 Avenue Jean Portalis, Tours, F-37200France

Personalised recommendations