Recognition of Long-Term Behaviors by Parsing Sequences of Short-Term Actions with a Stochastic Regular Grammar

  • Gerard Sanromà
  • Gertjan Burghouts
  • Klamer Schutte
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7626)


Human behavior understanding from visual data has applications such as threat recognition. A lot of approaches are restricted to limited time actions, which we call short-term actions. Long-term behaviors are sequences of short-term actions that are more extended in time. Our hypothesis is that they usually present some structure that can be exploited to improve recognition of short-term actions. We present an approach to model long-term behaviors using a syntactic approach. Behaviors to be recognized are hand-crafted into the model in the form of grammar rules. This is useful for cases when few (or no) training data is available such as in threat recognition. We use a stochastic parser so we handle noisy inputs. The proposed method succeeds in recognizing a set of predefined long-term interactions in the CAVIAR dataset. Additionally, we show how imposing prior knowledge about the structure of the long-term behavior improves the recognition of short-term actions with respect to standard statistical approaches.


long-term behavior stochastic context-free grammars human activity analysis visual surveillance 


  1. 1.
  2. 2.
    Blunsden, S., Andrade, E.L., Fisher, R.B.: Non Parametric Classification of Human Interaction. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007. LNCS, vol. 4478, pp. 347–354. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Brdiczka, O., Yuen, P.C., Zaidenberg, S., Reignier, P., Crowley, J.L.: Automatic acquisition of context models and its application to video surveillance. In: ICPR, pp. 1175–1178 (2006)Google Scholar
  4. 4.
    Burghouts, G.J., Marck, J.W.: Reasoning about threats: From observables to situation assessment. IEEE Transactions on Systems, Man, and Cybernetics, Part C 41(5), 608–616 (2011)CrossRefGoogle Scholar
  5. 5.
    Burghouts, G., Schutte, K.: Correlations between 48 human actions improve their detection. In: ICPR (2012)Google Scholar
  6. 6.
    Fernández-Caballero, A., Castillo, J.C., Rodríguez-Sánchez, J.M.: Human activity monitoring by local and global finite state machines. Expert Syst. Appl. 39(8), 6982–6993 (2012)CrossRefGoogle Scholar
  7. 7.
    Hays, D.G.: Chomsky hierarchy. In: Encyclopedia of Computer Science, pp. 210–211. John Wiley and Sons Ltd., ChichesterGoogle Scholar
  8. 8.
    Ivanov, Y.A., Bobick, A.F.: Recognition of visual activities and interactions by stochastic parsing. Pattern Anal. Mach. Intell. 22(8), 852–872 (2000)CrossRefGoogle Scholar
  9. 9.
    Kasteren, T.L., Englebienne, G., Kröse, B.J.: An activity monitoring system for elderly care using generative and discriminative models. Personal Ubiquitous Comput. 14(6), 489–498 (2010)CrossRefGoogle Scholar
  10. 10.
    Kitani, K.M., Sato, Y., Sugimoto, A.: Recovering the basic structure of human activities from a video-based symbol string. In: WMVC, p. 9 (2007)Google Scholar
  11. 11.
    Rubner, Y., Tomasi, C., Guibas, L.J.: The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vision 40(2), 99–121 (2000)zbMATHCrossRefGoogle Scholar
  12. 12.
    Stolcke, A.: An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Comput. Linguist. 21(2), 165–201 (1995)MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Gerard Sanromà
    • 1
  • Gertjan Burghouts
    • 1
  • Klamer Schutte
    • 1
  1. 1.TNOThe HagueThe Netherlands

Personalised recommendations