Multi-class Multi-scale Stacked Sequential Learning

  • Eloi Puertas
  • Sergio Escalera
  • Oriol Pujol
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6713)

Abstract

One assumption in supervised learning is that data is independent and identically distributed. However, this assumption does not hold true in many real cases. Sequential learning is that discipline of machine learning that deals with dependent data.

In this paper, we revise the Multi-Scale Sequential Learning approach (MSSL) for applying it in the multi-class case (MMSSL). We have introduced the ECOC framework in the MSSL base classifiers and a formulation for calculating confidence maps from the margins of the base classifiers. Another important contribution of this papers is the MMSSL compression approach for reducing the number of features in the extended data set. The proposed methods are tested on 5-class and 9-class image databases.

References

  1. 1.
    Allwein, E., Schapire, R., Singer, Y.: Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers. J. Machine Learning Research 1, 113–141 (2002)MATHGoogle Scholar
  2. 2.
    Dietterich, T.G., Bakiri, G.: Solving Multiclass Learning Problems via Error-Correcting Output Codes. J. Artificial Intelligence Research 2, 263–286 (1995)MATHGoogle Scholar
  3. 3.
    Dietterich, T.G.: Machine Learning for Sequential Data: A Review. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 15–30. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Dietterich, T.G., Ashenfelter, A., Bulatov, Y.: Training conditional random fields via gradient tree boosting. In: Proc. of the 21th ICML (2004)Google Scholar
  5. 5.
    Nilsson, N.J.: Learning Machines. McGraw-Hill, New York (1965)MATHGoogle Scholar
  6. 6.
    Cohen, W.W., de Carvalho, V.R.: Stacked sequential learning. In: IJCAI 2005, pp. 671–676 (2005)Google Scholar
  7. 7.
    McCallum, A., Freitag, D., Pereira, F.: Maximum entropy markov models for information extraction and segmentation. In: Proc. of ICML 2000, pp. 591–598 (2000)Google Scholar
  8. 8.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Annals of Statistics 28 (2000)CrossRefMATHGoogle Scholar
  9. 9.
    Wolpert, D.H.: Stacked generalization. Neural Networks 5(2), 241–259 (1992)CrossRefGoogle Scholar
  10. 10.
    Lafferty, J.D., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. of ICML 2001, pp. 282–289 (2001)Google Scholar
  11. 11.
    Pujol, O., Puertas, E., Gatta, C.: Multi-scale stacked sequential learning. In: Benediktsson, J.A., Kittler, J., Roli, F. (eds.) MCS 2009. LNCS, vol. 5519, pp. 262–271. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Korč, F., Förstner, W.: eTRIMS Image Database for Interpreting Images of Man-Made Scenes, TR-IGG-P-2009-01, University of Bonn (2009)Google Scholar
  13. 13.
    Boykov, Y., Funka-Lea, G.: Graph Cuts and Efficient N-D Image Segmentation. I. Journal of Computer Vision 70(2), 109–131 (2006)CrossRefGoogle Scholar
  14. 14.
    Escalera, S., Tax, D., Pujol, O., Radeva, P., Duin, R.: Subclass Problem-dependent Design of Error-Correcting Output Codes. IEEE T. in Pattern Analysis and Machine Intelligence 30(6), 1041–1054 (2008)CrossRefGoogle Scholar
  15. 15.
    Mottl, V., Dvoenko, S., Kopylov, A.: Pattern recognition in interrelated data: The problem, fundamental assumptions, recognition algorithms. In: Proc. of the 17th ICPR, Cambridge, UK vol. 1, pp. 188–191 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Eloi Puertas
    • 1
    • 2
  • Sergio Escalera
    • 1
    • 2
  • Oriol Pujol
    • 1
    • 2
  1. 1.Dept. Matemàtica Aplicada i AnàlisiUniversitat de BarcelonaBarcelonaSpain
  2. 2.Computer Vision Center, Campus UABBellaterraSpain

Personalised recommendations