Activity Detection Using Regular Expressions

  • Mattia Daldoss
  • Nicola Piotto
  • Nicola Conci
  • Francesco G. B. De Natale
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 158)


In this chapter we propose a novel method to analyze trajectories in surveillance scenarios by means of Context-Free Grammars (CFGs). Given a training corpus of trajectories associated to a set of actions, a preliminary processing phase is carried out to characterize the paths as sequences of symbols. This representation turns the numerical representation of the coordinates into a syntactical description of the activity structure, which is successively adopted to identify different behaviors through the CFG models. Such a modeling is the basis for the classification and matching of new trajectories versus the learned templates and it is carried out through a parsing engine that enables the online recognition of human activities. An additional module is provided to recover parsing errors (i.e., insertion, deletion, or substitution of symbols) and update the activity models previously learned. The proposed system has been validated in indoor, in an assisted living context, demonstrating good capabilities in recognizing activity patterns in different configurations, and in particular in presence of noise in the acquired trajectories, or in case of concatenated and nested actions.


Activity analysis Context-free grammar Regular expressions Activity classification Anomaly detection 



The research has been developed under the project ACube funded by the Provincia Autonoma di Trento (Italy).


  1. 1.
    Adriaans PW, Vervoort M (2002) The emile 4.1 grammar induction toolbox. In: International colloquium on grammatical inference, Springer-Verlag GmbH, pp 293–295Google Scholar
  2. 2.
    Berndt D, Clifford J (1994) Using dynamic time warping to find patterns in time series. In: Workshop on knowledge discovery and databases, pp 229–248Google Scholar
  3. 3.
    Bilmes J (1998) A gentle tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Technical ReportGoogle Scholar
  4. 4.
    Daldoss M, Piotto N, Conci N, De Natale FGB (2010) Learning and matching human activities using regular expressions. In: IEEE international conference on image processing, pp 4681–4684Google Scholar
  5. 5.
    Das G, Gunopulos D, Mannila H (1997) Finding similar time series. In: Proceedings of the European symposium on principles of data mining and knowledge discovery, Springer-Verlag GmbH, pp 88–100Google Scholar
  6. 6.
    Duong TV, Bui H, Phung DQ, Venkatesh S (2005) Activity recognition and abnormality detection with the switching hidden semi-markov model. In: IEEE international conference on computer vision and pattern recognition, vol 1, pp 838–845Google Scholar
  7. 7.
    Earley J (1970) An efficient context-free parsing algorithm. Commun ACM 13(2):94–102MATHCrossRefGoogle Scholar
  8. 8.
    Hamid R, Maddi S, Johnson A, Bobick A, Essa I, Isbell C (2009) A novel sequence representation for unsupervised analysis of human activities. J Artif Intell 173(14):1221–1244MathSciNetCrossRefGoogle Scholar
  9. 9.
    Ivanov YA, Bobick A (2000) Recognition of visual activities and interactions by stochastic parsing. IEEE Trans Pattern Anal Mach Intell 22(8):852–872CrossRefGoogle Scholar
  10. 10.
    Joo SW, Chellappa R (2006) Attribute grammar-based event recognition and anomaly detection. In: IEEE international conference on computer vision and pattern recognition workshop, pp 107–107Google Scholar
  11. 11.
    Knuth DE (1968) Semantics of context-free languages. Theory Comput Syst 2(2):127–145MathSciNetMATHGoogle Scholar
  12. 12.
    Laxton B, Lim J, Kriegman D (2007) Leveraging temporal, contextual and ordering constraints for recognizing complex activities in video. In: IEEE international conference on computer vision and pattern recognition, pp 1–8Google Scholar
  13. 13.
    Morris B, Trivedi M (2008) A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans Circuits Syst Video Technol 18(8):1114–1127CrossRefGoogle Scholar
  14. 14.
    Minnen D, Essa I, Starner T (2003) Expectation grammars: leveraging high-level expectations for activity recognition. In: IEEE international conference on computer vision and pattern recognition, vol 2, pp 626–632Google Scholar
  15. 15.
    Moore D, Essa I (2001) Recognizing multitasked activities using stochastic context-free grammar. In: Proceedings of AAAI conferenceGoogle Scholar
  16. 16.
    Nguyen NT, Phung DQ, Venkatesh S, Bui H (2005) Learning and detecting activities from movement trajectories using the hierarchical hidden markov models. In: IEEE international conference on computer vision and pattern recognition, vol. 2, pp 955–960Google Scholar
  17. 17.
    Piciarelli C, Micheloni C, Foresti G (2008) Trajectory-based anomalous event detection. IEEE Trans Circuits Syst Video Technol 18(11):1544–1554CrossRefGoogle Scholar
  18. 18.
    Piotto N, Conci N, De Natale F (2009) Syntactic matching of trajectories for ambient intelligence applications. IEEE Trans Multimedia 11(7):1266–1275CrossRefGoogle Scholar
  19. 19.
    Prati A, Calderara S, Cucchiara R (2008) Using circular statistics for trajectory shape analysis. In: IEEE international conference on computer vision and pattern recognition, pp 1–8Google Scholar
  20. 20.
    Stauffer C, Grimson W (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757CrossRefGoogle Scholar
  21. 21.
    Zhang Z, Tan T, Huang K (2011) An extended grammar system for learning and recognizing complex visual events. IEEE Trans Pattern Anal Mach Intell 33(2):240–255Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Mattia Daldoss
    • 1
  • Nicola Piotto
    • 1
  • Nicola Conci
    • 1
  • Francesco G. B. De Natale
    • 1
  1. 1.Multimedia Signal Processing and Understanding LabDISI—University of TrentoTrentoItaly

Personalised recommendations