Fractional Programming Weighted Decoding for Error-Correcting Output Codes

  • Firat Ismailoglu
  • I. G. Sprinkhuizen-Kuyper
  • Evgueni Smirnov
  • Sergio Escalera
  • Ralf Peeters
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9132)

Abstract

In order to increase the classification performance obtained using Error-Correcting Output Codes designs (ECOC), introducing weights in the decoding phase of the ECOC has attracted a lot of interest. In this work, we present a method for ECOC designs that focuses on increasing hypothesis margin on the data samples given a base classifier. While achieving this, we implicitly reward the base classifiers with high performance, whereas punish those with low performance. The resulting objective function is of the fractional programming type and we deal with this problem through the Dinkelbach’s Algorithm. The conducted tests over well known UCI datasets show that the presented method is superior to the unweighted decoding and that it outperforms the results of the state-of-the-art weighted decoding methods in most of the performed experiments.

References

  1. 1.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, Hoboken (2004)CrossRefGoogle Scholar
  2. 2.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1, 113–141 (2001)MATHMathSciNetGoogle Scholar
  3. 3.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. (JAIR) 2, 263–286 (1995)MATHGoogle Scholar
  4. 4.
    Kong, E.B., Dietterich, T.G.: Error-correcting output coding corrects bias and variance. In: ICML, pp. 313–321 (1995)Google Scholar
  5. 5.
    Guruswami, V., Sahai, A.: Multiclass learning, boosting, and error-correcting codes. In: 12th Annual Conference Computational Learning Theory, Santa Cruz, California, pp. 145–155 (1999)Google Scholar
  6. 6.
    Schapire, R.E., Freund, Y.: Boosting: Foundations and Algorithms, vol. 1. MIT Press, Cambridge (2012)Google Scholar
  7. 7.
    Escalera, S., Pujol, O., Radeva P.: Loss-weighted decoding for error-correcting output codes. In: International Conference on Computer Vision Theory and Applications, Madeira, Portugal (2008)Google Scholar
  8. 8.
    Smith, R.S., Windeatt, T.: Class-separability weighting and bootstrapping in error-correcting output code ensembles. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 185–194. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  9. 9.
    Crammer, K., Gilad-Bachrach, R., Navot, A., Tishby, N.: Margin analysis of the LVQ algorithm. In: Proceedings of 17th Conference on Neural Information Processing Systems (2002)Google Scholar
  10. 10.
    Escalera, S., Pujol, O., Radeva, P.: On the decoding process in ternary error-correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 120–134 (2010)CrossRefGoogle Scholar
  11. 11.
    Bajalinov, E.B.: Linear-Fractional Programming: Theory, Methods, Applications and Software, 1st edn. Kluwer Academic Publishers, New York (2003) CrossRefGoogle Scholar
  12. 12.
    Zhang, X., Wu, J., Chen, Z., Lv, P.: Optimized weighted decoding for error-correcting output codes. In: IEEE International Conference on Acoustics, Speech and Signal Processing, Kyoto, Japan (2012)Google Scholar
  13. 13.
    Parades, R., Vidal, E.: A class-dependent weighted dissimilarity measure for nearest neighbor classification problems. Pattern Recogn. Lett. 21(12), 1027–1036 (2000)CrossRefGoogle Scholar
  14. 14.
    Sun, Y., Todorovic, S., Li, J., Wu, D.: Unifying the error-correcting output code AdaBoost within the margin framework. In: 22nd ICML, Bonn, Germany (2005)Google Scholar
  15. 15.
    Sniedovich, M.: Dynamic Programming Foundations and Principles, 2nd edn. CRC Press, USA (2011) MATHGoogle Scholar
  16. 16.
    Gugat, M.: Prox-regularization methods for generalized fractional programming. J. Optim. Theory Appl. 99(3), 691–722 (1998)CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Domeniconi, C., Gunopulos, D., Ma, S., Yan, B., Al-Razgan, M., Papadopoulos, D.: Locally adaptive metrics for clustering high dimensional data. J. Data Min. Knowl. Discov. 14(1), 63–67 (2007)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Smirnov, E., Moed, M., Kuyper, I.: Minimally-Sized Balanced Decomposition Schemes for Multi-class Classification. Ensembles in Machine Learning Applications. Springer, Berlin (2011) CrossRefGoogle Scholar
  19. 19.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011)CrossRefGoogle Scholar
  20. 20.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. JMLR 7, 1–30 (2006)MATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Firat Ismailoglu
    • 1
  • I. G. Sprinkhuizen-Kuyper
    • 2
  • Evgueni Smirnov
    • 1
  • Sergio Escalera
    • 3
  • Ralf Peeters
    • 1
  1. 1.Maastricht UniversityDepartment of Knowledge EngineeringMaastrichtThe Netherlands
  2. 2.Radboud University NijmegenDonders Institute for Brain, Cognition and Behaviour, Centre for CognitionNijmegenThe Netherlands
  3. 3.University of Barcelona and Computer Vision CenterBarcelonaSpain

Personalised recommendations