A Cost-Sensitive Paradigm for Multiclass to Binary Decomposition Schemes

  • Claudio Marrocco
  • Francesco Tortorella
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)


An established technique to face a multiclass categorization problem is to reduce it into a set of two-class problems. To this aim, the main decomposition schemes employed are one vs. one, one vs. all and Error Correcting Output Coding. A point not yet considered in the research is how to apply these methods to a cost-sensitive classification that represents a significant aspect in many real problems. In this paper we propose a novel method which, starting from the cost matrix for the multi-class problem and from the code matrix employed, extracts a cost matrix for each of the binary subproblems induced by the coding matrix. In this way, it is possible to tune the single two-class classifier according to the cost matrix obtained and achieve an output from all the dichotomizers which takes into account the requirements of the original multi-class cost matrix. To evaluate the effectiveness of the method, a large number of tests has been performed on real data sets. The experiments results have shown a significant improvement in terms of classification cost, specially when using the ECOC scheme.


False Negative Rate Decomposition Scheme Cost Matrix True Negative Rate Code Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Hastie, T., Tibshirani, R.: Classification by Pairwise Coupling. The Annals of Statistics 26, 451–471 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Dietterich, T.G., Bakiri, G.: Solving Multiclass Learning Problems via Error Correcting Output Codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  3. 3.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers. Journal of Machine Learning Research 1, 113–141 (2000)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Windeatt, T., Ghaderi, R.: Coding and Decoding Strategies for Multi Class Learning Problems. Information Fusion 4, 11–21 (2003)CrossRefGoogle Scholar
  5. 5.
    Tortorella, F.: An Empirical Comparison of In-Learning and Post-Learning Optimization Schemes for Tuning the Support Vector Machines in Cost-Sensitive Applications. In: Proc. 12th Int. Conf. on Image Anal. and Proc., pp. 560–565. IEEE Computer Society Press, Los Alamitos (2003)CrossRefGoogle Scholar
  6. 6.
    Blake, C., Keogh, E., Merz, C.J.: UCI Repository of Machine Learning Databases (1998),
  7. 7.
    Joachims, T.: Making Large-Scale SVM Learning Practical. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods, pp. 169–184. MIT Press, Cambridge (1999)Google Scholar
  8. 8.
    NIST/SEMATECH e-Handbook of Statistical Methods (2003),
  9. 9.
    Margineantu, D.D., Dietterich, T.G.: Bootstrap Methods for the Cost-Sensitive Evaluation of Classifiers. In: Proc. Int. Conf. Machine Learning ICML 2000, pp. 582–590 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Claudio Marrocco
    • 1
  • Francesco Tortorella
    • 1
  1. 1.Dipartimento di Automazione, Elettromagnetismo, Ingegneria dell’Informazione e Matematica IndustrialeUniversità degli Studi di CassinoCassino (FR)Italy

Personalised recommendations