Coding Theory Tools for Improving Recognition Performance in ECOC Systems

  • Claudio Marrocco
  • Paolo Simeone
  • Francesco Tortorella
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7872)


Error-correcting output coding (ECOC) is nowadays an established technique to build polychotomous classification systems by aggregating highly efficient dichotomizers. This approach has exhibited good classification performance and generalization capabilities in many practical applications. In this field much work has been devoted to study new solutions both for the coding and the decoding phase, but little attention has been paid to the algebraic tools typically employed in the Coding Theory, which could provide an ECOC design approach based on robust theoretical foundations. In this paper we propose an ECOC classification system based on Low Density Parity Check (LDPC) Codes, a well known technique in Coding Theory. Such framework is particularly suitable to define an ECOC system that employs dichotomizers provided of a reject option. The experiments on some public data sets have demonstrated that, in this way, the ECOC system can reach good recognition rates when a suitable reject level is imposed to the dichotomizers.


ECOC reject option LDPC coding theory ensemble learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2000)MathSciNetGoogle Scholar
  2. 2.
    Alpaydin, E., Mayoraz, E.: Learning error-correcting output codes from data. In: Ninth International Conference on Artificial Neural Networks, ICANN 1999, Conf. Publ. No. 470, vol. 2, pp. 743–748 (1999)Google Scholar
  3. 3.
    Bautista, M.Á., Escalera, S., Baró, X., Radeva, P., Vitrià, J., Pujol, O.: Minimal design of error-correcting output codes. Pattern Recognition Letters 33(6), 693–702 (2012)CrossRefGoogle Scholar
  4. 4.
    Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. In: Cesa-Bianchi, N., Goldman, S.A. (eds.) COLT, pp. 35–46. Morgan Kaufmann (2000)Google Scholar
  5. 5.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  6. 6.
    Escalera, S., Pujol, O., Radeva, P.: On the decoding process in ternary error-correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 120–134 (2010)CrossRefGoogle Scholar
  7. 7.
    Escalera, S., Tax, D.M.J., Pujol, O., Radeva, P., Duin, R.P.W.: Subclass problem-dependent design for error-correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1041–1054 (2008)CrossRefGoogle Scholar
  8. 8.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
  9. 9.
    Gallager, R.G.: Low density parity-check codes. MIT Press (1963)Google Scholar
  10. 10.
    Joachims, T.: Making large-scale SVM learning practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning, ch. 11. MIT Press, Cambridge (1999)Google Scholar
  11. 11.
    Kong, E.B., Dietterich, T.G.: Error-correcting output coding corrects bias and variance. In: ICML, pp. 313–321 (1995)Google Scholar
  12. 12.
    Marrocco, C., Simeone, P., Tortorella, F.: Embedding reject option in ECOC through LDPC codes. In: Haindl, M., Kittler, J., Roli, F. (eds.) MCS 2007. LNCS, vol. 4472, pp. 333–343. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  13. 13.
    Pietraszek, T.: On the use of ROC analysis for the optimization of abstaining classifiers. Machine Learning 68(2), 137–169 (2007)CrossRefGoogle Scholar
  14. 14.
    Pishro-Nik, H., Fekri, F.: On decoding of low-density parity-check codes over the binary erasure channel. IEEE Trans. Inf. Theor. 50(3), 439–454 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Pujol, O., Radeva, P., Vitrià, J.: Discriminant ECOC: A heuristic method for application dependent design of error correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 28(6), 1007–1012 (2006)CrossRefGoogle Scholar
  16. 16.
    Richardson, T.J., Urbanke, R.: Modern Coding Theory. Cambridge University Press (2008)Google Scholar
  17. 17.
    Shokrollahi, A.: An introduction to low-density parity-check codes. In: Khosrovshahi, G.B., Shokoufandeh, A., Shokrollahi, A. (eds.) Theoretical Aspects of Computer Science 2000. LNCS, vol. 2292, pp. 175–197. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. 18.
    Tanner, R.M.: A Recursive Approach to Low Complexity Codes. IEEE Transactions on Information Theory 27(5), 533–547 (1981)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Tapia, E., Bulacio, P., Angelone, L.: Recursive ECOC classification. Pattern Recognition Letters 31(3), 210–215 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Claudio Marrocco
    • 1
  • Paolo Simeone
    • 1
  • Francesco Tortorella
    • 1
  1. 1.DIEIUniversità degli Studi di CassinoCassinoItalia

Personalised recommendations