Softmax Regression for ECOC Reconstruction

  • Roberto D’Ambrosio
  • Giulio Iannello
  • Paolo Soda
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8156)


Classification by binary decomposition is a well-known method to solve multiclass classification tasks since a large number of algorithms were designed for binary classification. Once the polychotomy has been decomposed into several dichotomies, the decisions of binary learners on a test sample are aggregated by a reconstruction rule to set the final multiclass label. In this context, this paper presents a reconstruction rule based on softmax regression which considers the reconstruction task as a new classification problem. To this aim, as second-order features we use both the crisp labels and the reliabilities of binary decisions. Six heterogeneous datasets and three different classification architectures have been used to test our method, whose performance favorably compare with those provided by other three reconstruction rules both in terms of global accuracy and geometric mean of accuracies.


Support Vector Machine Minority Class Heterogeneous Dataset Binary Decomposition Soft Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2001)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Cordella, L.P., Foggia, P.: Reliability parameters to improve combination strategies in multi-expert systems. Pattern Analysis and Applications 2, 205–214 (1999)CrossRefGoogle Scholar
  3. 3.
    Cordella, L.P., Foggia, P., et al.: Reliability parameters to improve combination strategies in multi-expert systems. Pattern Analysis & Applications 2(3), 205–214 (1999)CrossRefGoogle Scholar
  4. 4.
    D’Ambrosio, R., Iannello, G., Soda, P.: A one-per-class reconstruction rule for class imbalance learning. In: 21st Int. Conf. on Pattern Recognition, pp. 1310–1313. IEEE (2012)Google Scholar
  5. 5.
    D’Ambrosio, R., Soda, P.: Polichotomies on imbalanced domains by one-per-class compensated reconstruction rule. In: Gimel’farb, G., Hancock, E., Imiya, A., Kuijper, A., Kudo, M., Omachi, S., Windeatt, T., Yamada, K. (eds.) SSPR & SPR 2012. LNCS, vol. 7626, pp. 301–309. Springer, Heidelberg (2012)Google Scholar
  6. 6.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263 (1995)zbMATHGoogle Scholar
  7. 7.
    Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Machine Learning-International Workshop then Conference, pp. 148–156 (1996)Google Scholar
  8. 8.
    Fürnkranz, J.: Round robin classification. J. of Machine Learning Research 2, 721–747 (2002)zbMATHGoogle Scholar
  9. 9.
    Hsu, C.W., Lin, C.J.: A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 13(2), 415–425 (2002)CrossRefGoogle Scholar
  10. 10.
    Iannello, G., Percannella, G., Sansone, C., Soda, P.: On the use of classification reliability for improving performance of the one-per-class decomposition method. Data & Knowledge Engineering 68, 1398–1410 (2009)CrossRefGoogle Scholar
  11. 11.
    Jelonek, J., Stefanowski, J.: Experiments on solving multiclass learning problems by n 2 classifier. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 172–177. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  12. 12.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)CrossRefGoogle Scholar
  13. 13.
    Kumar, S., Ghosh, J., Crawford, M.M.: Hierarchical fusion of multiple classifiers for hyperspectral data analysis. Pattern Analysis & Applications 5(2), 210–220 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Masulli, F., Valentini, G.: Comparing decomposition methods for classication. In: Fourth International Conference on Knowledge-Based Intelligent Engineering Systems & Allied Technologies, KES 2000, pp. 788–791 (2000)Google Scholar
  15. 15.
    Moreira, M., Mayoraz, E.: Improved pairwise coupling classification with correcting classifiers. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 160–171. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  16. 16.
    Platt, J.: Probabilistic output for support vector machines and comparisons to regularize likelihood methods. Advanced in Large Margin Classifiers. MIT Press (2000)Google Scholar
  17. 17.
    Rajan, S., Ghosh, J.: An empirical comparison of hierarchical vs. two-level approaches to multiclass problems. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 283–292. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Rtsch, G., Smola, A.J., Mika, S.: Adapting codes and embeddings for polychotomies (2003)Google Scholar
  19. 19.
    Selman, B., Levesque, H., Mitchell, D., et al.: A new method for solving hard satisfiability problems. In: 10th National Conference on Artificial Intelligence, pp. 440–446 (1992)Google Scholar
  20. 20.
    Shiraishi, Y., Fukumizu, K.: Statistical approaches to combining binary classifiers for multi-class classification. Neurocomputing 74(5), 680–688 (2011)CrossRefGoogle Scholar
  21. 21.
    Soda, P.: A multi-objective optimisation approach for class-imbalance learning. Pattern Recognition 44, 1801–1810 (2011)CrossRefzbMATHGoogle Scholar
  22. 22.
    Stefanowski, J.: Multiple and hybrid classifiers. pp. 174–188 (2001)Google Scholar
  23. 23.
    Xu, L., Krzyzak, A., Suen, C.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics 22(3), 418–435 (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Roberto D’Ambrosio
    • 1
  • Giulio Iannello
    • 1
  • Paolo Soda
    • 1
  1. 1.Integrated Research CentreUniversitá Campus Bio-Medico di RomaRomeItaly

Personalised recommendations