On the Design of Low Redundancy Error-Correcting Output Codes

Part of the Studies in Computational Intelligence book series (SCI, volume 373)


The classification of large number of object categories is a challenging trend in the Pattern Recognition field. In the literature, this is often addressed using an ensemble of classifiers . In this scope, the Error-Correcting Output Codes framework has demonstrated to be a powerful tool for combining classifiers. However, most of the state-of-the-art ECOC approaches use a linear or exponential number of classifiers, making the discrimination of a large number of classes unfeasible. In this paper, we explore and propose a compact design of ECOC in terms of the number of classifiers. Evolutionary computation is used for tuning the parameters of the classifiers and looking for the best compact ECOC code configuration. The results over several public UCI data sets and different multi-class Computer Vision problems show that the proposed methodology obtains comparable (even better) results than the state-of-the-art ECOC methodologies with far less number of dichotomizers.


Machine Learn Research Binary Problem Multiclass Problem Codeword Length Face Category 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alamús, R., Baron, A., Bosch, E., Casacuberta, J., Miranda, J., Pla, M., Sànchez, S., Serra, A., Talaya, J.: On the accuracy and performance of the GeoMobil system. In: Proc. the 20th Congress Int. Soc. Photogrammetry and Remote Sens., Istanbul, Turkey, pp. 262–267 (2004)Google Scholar
  2. 2.
    Allwein, E., Schapire, R., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. J. Machine Learning Research 1, 113–141 (2002)MathSciNetGoogle Scholar
  3. 3.
    Asuncion, A., Newman, D.J.: UCI machine learning repository,
  4. 4.
    Baluja, S., Caruana, R.: Removing the genetics from the standard genetic algorithm. In: Prieditis, A., Russel, S. (eds.) Proc. the 12th Int. Conf. Machine Learning, Tahoe City, CA, pp. 38–46. Morgan Kaufmann, San Francisco (1995)Google Scholar
  5. 5.
    Crammer, K., Singer, Y.: On the learnability and design of output codes for multi-class problems. Machine Learning 47, 201–233 (2002)CrossRefzbMATHGoogle Scholar
  6. 6.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Machine Learning Research 7, 1–30 (2006)MathSciNetGoogle Scholar
  7. 7.
    Dietterich, T., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intel. Research 2, 263–286 (1995)zbMATHGoogle Scholar
  8. 8.
    Dietterich, T., Kong, E.: Error-correcting output codes corrects bias and variance. In: Prieditis, A., Russell, S. (eds.) Proc. 12th Int. Conf. Machine Learning, Tahoe City, CA, pp. 313–321. Morgan Kaufmann, San Francisco (1995)Google Scholar
  9. 9.
    Escalera, S., Pujol, O., Radeva, P.: On the decoding process in ternary error-correcting output codes. IEEE Trans. Pattern Analysis and Machine Intel. 32, 120–134 (2010)CrossRefGoogle Scholar
  10. 10.
    Escalera, S., Pujol, O., Radeva, P.: Error-correcting output codes library. J. Machine Learning Research 11, 661–664 (2010)Google Scholar
  11. 11.
    Fornés, A., Lladós, J., Sánchez, G.: Primitive segmentation in old handwritten music scores. In: Liu, W., Lladós, J. (eds.) GREC 2005. LNCS, vol. 3926, pp. 279–290. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. The Annals of Statistics 38, 337–374 (1998)MathSciNetGoogle Scholar
  13. 13.
    Hastie, T., Tibshirani, R.: Classification by pairwise grouping. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) Advances in Neural Inf. Proc. Syst., vol. 10, pp. 507–513. MIT Press, Cambridge (1998)Google Scholar
  14. 14.
    Holland, J.H.: Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. MIT Press, Cambridge (1975)Google Scholar
  15. 15.
    Huang, G.B., Ramesh, M., Berg, T., Miller, E.L.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report, pp. 7–49. University of Massachusets, Amherst (2007)Google Scholar
  16. 16.
    Hwang, W., Weng, J., Zhang, Y.: Candid covariance-free incremental principal component analysis. IEEE Trans. Pattern Analysis and Machine Intel. 25, 1034–1040 (2003)CrossRefGoogle Scholar
  17. 17.
    Lorena, A.-C., de Carvalho, A.C.P.L.F.: Evolutionary tuning of svm parameter values in multiclass problems. Neurocomputing 71, 3326–3334 (2008)CrossRefGoogle Scholar
  18. 18.
    Martínez, A., Benavente, R.: The AR face database. Computer Vision Center Technical Report 24, University of Barcelona (1998)Google Scholar
  19. 19.
    Nilsson, N.J.: Learning machines: Foundations of trainable pattern-classifying systems. McGraw-Hill, New York (1965)zbMATHGoogle Scholar
  20. 20.
  21. 21.
    Passerini, A., Pontil, M., Frasconi, P.: New results on error correcting output codes of kernel machines. IEEE Trans. Neural Networks 15, 45–54 (2004)CrossRefGoogle Scholar
  22. 22.
    Pujol, O., Radeva, P., Vitrià, J.: Discriminant ECOC: A heuristic method for application dependent design of error correcting output codes. IEEE Trans. Pattern Analysis and Machine Intel. 28, 1001–1007 (2006)CrossRefGoogle Scholar
  23. 23.
    Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Machine Learning Research 5, 101–141 (2004)MathSciNetGoogle Scholar
  24. 24.
    Utschick, W., Weichselberger, W.: Stochastic organization of output codes in multiclass learning problems. Neural Computation 13, 1065–1102 (2004)CrossRefGoogle Scholar
  25. 25.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Applied Math and Analysis DepartmentUniversity of BarcelonaBarcelonaSpain
  2. 2.Computer Vision CenterAutonomous University of BarcelonaCerdanyolaSpain
  3. 3.Universitat Oberta de CatalunyaBarcelonaSpain

Personalised recommendations