Decremental Learning of Evolving Fuzzy Inference Systems: Application to Handwritten Gesture Recognition

  • Manuel Bouillon
  • Eric Anquetil
  • Abdullah Almaksour
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7988)

Abstract

This paper tackles the problem of incremental and decremental learning of an evolving and customizable fuzzy inference system for classification. We explain the interest of integrating a forgetting capacity in such an evolving system to improve its performances in changing environments. In this paper, we describe two decremental learning strategies to introduce a forgetting capacity in evolving fuzzy inference systems. Both techniques use a sliding window to introduce forgetting in the optimization process of fuzzy rules conclusions. The first approach is based on a downdating technique of least squares solutions for unlearning old data. The second integrates differed directional forgetting in the covariance matrices used in the recursive least square algorithm. These techniques are first evaluated on handwritten gesture recognition tasks in changing environments. They are also evaluated on some well-known classification benchmarks. In particular, it is shown that decremental learning allow to adapt to concept drifts. It is also demonstrated that decremental learning is necessary to maintain the system capacity of learning new classes over time, making decremental learning essential for the life-time use of an evolving and customizable classification system.

Keywords

Online Classification Handwriting Recognition Incremental Learning Decremental Learning Evolving Fuzzy Inference System Recursive Least Squares Concept Drifts Forgetting 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Almaksour, A., Anquetil, E.: Improving premise structure in evolving takagi-sugeno neuro-fuzzy classifiers. Evolving Systems 2, 25–33 (2011)CrossRefGoogle Scholar
  2. 2.
    Angelov, P., Zhou, X.: Evolving fuzzy-rule-based classifiers from data streams. IEEE Transactions on Fuzzy Systems 16(6), 1462–1475 (2008)CrossRefGoogle Scholar
  3. 3.
    Angelov, P., Filev, D.: An approach to online identification of takagi-sugeno fuzzy models. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetic 34(1), 484–498 (2004)CrossRefGoogle Scholar
  4. 4.
    Delaye, A., Anquetil, E.: HBF49 feature set: A first unified baseline for online symbol recognition. Pattern Recognition 46(1), 117–130 (2013)CrossRefGoogle Scholar
  5. 5.
    Fortescue, T., Kershenbaum, L., Ydstie, B.: Implementation of self-tuning regulators with variable forgetting factors. Automatica 17(6), 831–835 (1981)CrossRefGoogle Scholar
  6. 6.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml
  7. 7.
    Gama, J., Sebastião, R., Rodrigues, P.P.: Issues in evaluation of stream learning algorithms. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 329–338 (2009)Google Scholar
  8. 8.
    Hägglund, T.: New estimation techniques for adaptive control (1983)Google Scholar
  9. 9.
    Haykin, S.O.: Adaptive Filter Theory, 4th edn. Prentice Hall (2001)Google Scholar
  10. 10.
    Kulhavy, R., Zarrop, M.: On a general concept of forgetting. International Journal of Control 58(4), 905–924 (1993)MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Liavas, A., Regalia, P.: On the numerical stability and accuracy of the conventional recursive least squares algorithm. Trans. Signal Processing 47(1), 88–96 (1999)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Lughofer, E.: Evolving fuzzy models: incremental learning, interpretability, and stability issues, applications. VDM Verlag Dr. Müller (2008)Google Scholar
  13. 13.
    Renau-Ferrer, N., Li, P., Delaye, A., Anquetil, E.: The ILGDB database of realistic pen-based gestural commands. In: International Conference on Pattern Recognition (ICPR 2012), tsukuba, Japan (November 2012)Google Scholar
  14. 14.
    Salgado, M.E., Goodwin, G.C., Middleton, R.H.: Modified least squares algorithm incorporating exponential resetting and forgetting. International Journal of Control 47(2), 477–491 (1988)MATHCrossRefGoogle Scholar
  15. 15.
    Takagi, T., Sugeno, M.: Fuzzy Identification of Systems and Its Applications to Modeling and Control. IEEE Transactions on Systems, Man, and Cybernetics 15(1), 116–132 (1985)MATHCrossRefGoogle Scholar
  16. 16.
    Viard-Gaudin, C., Lallican, P.M., Binter, P., Knerr, S.: The IRESTE On/Off (IRONOFF) dual handwriting database. In: Proceedings of the Fifth International Conference on Document Analysis and Recognition, ICDAR 1999, pp. 455–458 (1999)Google Scholar
  17. 17.
    Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23(1), 69–101 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Manuel Bouillon
    • 1
    • 2
    • 3
  • Eric Anquetil
    • 1
    • 2
    • 3
  • Abdullah Almaksour
    • 1
    • 2
    • 3
  1. 1.Université Européenne de BretagneFrance
  2. 2.INSA de RennesRennesFrance
  3. 3.IRISARennesFrance

Personalised recommendations