Semantic Video Content Analysis

  • Massimiliano Albanese
  • Pavan Turaga
  • Rama Chellappa
  • Andrea Pugliese
  • V. S. Subrahmanian
Part of the Studies in Computational Intelligence book series (SCI, volume 287)


In recent years, there has been significant interest in the area of automatically recognizing activities occurring in a camera’s field of view and detecting abnormalities. The practical applications of such a system could include airport tarmac monitoring, or monitoring of activities in secure installations, to name a few. The difficulty of the problem is compounded by several factors: detection of primitive actions in spite of changes in illumination, occlusions and noise; complexmultiagent interaction;mapping of higher-level activities to lower-level primitive actions; variations in which the same semantic activity can be performed. In this chapter, we develop a theory of semantic activity analysis that addresses each of these issues in an integrated manner. Specifically, we discuss ontological representations of knowledge of a domain, integration of domain knowledge and statistical models for achieving semantic mappings, definition of logical languages to describe activities, and design of frameworks which integrate all the above aspects in a coherent way, thus laying the foundations of effective Semantic Video Content Analysis systems.


Video Sequence Activity Recognition Probability Threshold Start Node Alert Level 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akdemir, U., Turaga, P., Chellappa, R.: An ontology based approach for activity recognition from video. In: Proc. of the 16th ACM Intl. Conf. on Multimedia (MM 2008), pp. 709–712 (2008)Google Scholar
  2. 2.
    Albanese, M., Chellappa, R., Moscato, V., Picariello, A., Subrahmanian, V.S., Turaga, P., Udrea, O.: A constrained probabilistic petri net framework for human activity detection in video. IEEE Transactions on Multimedia 10(8), 1429–1443 (2008)CrossRefGoogle Scholar
  3. 3.
    Albanese, M., Moscato, V., Picariello, A., Subrahmanian, V.S., Udrea, O.: Detecting stochastically scheduled activities in video. In: Proc. of the 20th Intl. Joint Conf. on Artificial Intelligence (IJCAI 2007), pp. 1802–1807 (2007)Google Scholar
  4. 4.
    Albanese, M., Pugliese, A., Subrahmanian, V.S., Udrea, O.: MAGIC: A multi-activity graph index for activity detection. In: Proc. of the IEEE Intl. Conf. on Information Reuse and Integration (IRI 2007), pp. 267–278 (2007)Google Scholar
  5. 5.
    Avrahami-Zilberbrand, D., Kaminka, G., Zarosim, H.: Fast and complete symbolic plan recognition: Allowing for duration, interleaved execution, and lossy observations. In: Proc. of the AAAI Workshop on Modeling Others from Observations, MOO 2005 (2005)Google Scholar
  6. 6.
    Chen, D., Yang, J., Wactlar, H.D.: Towards automatic analysis of social interaction patterns in a nursing home environment from video. In: Proc. of the 6th ACM SIGMM Intl. Workshop on Multimedia Information Retrieval (MIR 2004), pp. 283–290 (2004)Google Scholar
  7. 7.
    Chen, S.M., Ke, J.S., Chang, J.F.: Knowledge representation using fuzzy petri nets. IEEE Transactions on Knowledge and Data Engineering 2(3), 311–319 (1990)CrossRefGoogle Scholar
  8. 8.
    David, R., Alla, H.: Petri nets for modeling of dynamic systems a survey. Automatica 30(2), 175–202 (1994)Google Scholar
  9. 9.
    Duong, T.V., Bui, H.H., Phung, D.Q., Venkatesh, S.: Activity recognition and abnormality detection with the switching hidden semi-markov model. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 838–845 (2005)Google Scholar
  10. 10.
    Georis, B., Maziere, M., Brémond, F., Thonnat, M.: A video interpretation platform applied to bank agency monitoring. In: IEE Intelligent Distributed Surveilliance Systems (IDSS-04), pp. 46–50 (2004)Google Scholar
  11. 11.
    Gruber, T.R.: Toward principles for the design of ontologies used for knowledge sharing. Intl. Journal of Human-Computer Studies 43(5-6), 907–928 (1995)CrossRefGoogle Scholar
  12. 12.
    Guler, S., Burns, J.B., Hakeem, A., Sheikh, Y., Shah, M., Thonnat, M., Bremond, F., Maillot, N., Vu, T.V., Haritaoglu, I., Chellappa, R., Akdemir, U., Davis, L.: An ontology of video events in the physical security and surveillance domain (2004),
  13. 13.
    Hakeem, A., Shah, M.: Ontology and taxonomy collaborated framework for meeting classification. In: Proc. of the 17th Intl. Conf. on Pattern Recognition (ICPR 2004), vol. 4, pp. 219–222. IEEE Computer Society, Los Alamitos (2004)CrossRefGoogle Scholar
  14. 14.
    Hamid, R., Huang, Y., Essa, I.: ARGMode – activity recognition using graphical models. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition Workshop (CVPRW 2003), vol. 4, pp. 38–43 (2003)Google Scholar
  15. 15.
    Hobbs, J., Nevatia, R., Bolles, B.: An ontology for video event representation. In: Proc. of the 2004 IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW 2004), p. 119 (2004)Google Scholar
  16. 16.
    Luhr, S., Bui, H.H., Venkatesh, S., West, G.A.W.: Recognition of human activity through hierarchical stochastic learning. In: Proc. of the First IEEE Intl. Conf. on Pervasive Computing and Communications (PCC 2003), pp. 416–422 (2003)Google Scholar
  17. 17.
    Marhasev, E., Hadad, M., Kaminka, G.A.: Non-stationary hidden semi-markov models in activity recognition. In: Proc. of the AAAI Workshop on Modeling Others from Observations, MOO 2006 (2006)Google Scholar
  18. 18.
    Marsan, M.A., Balbo, G., Chiola, G., Conte, G., Donatelli, S., Franceschinis, G.: An introduction to generalized stochastic petri nets. Microelectronics and Reliability 31(4), 699–725 (1991)CrossRefGoogle Scholar
  19. 19.
    Murata, T.: Petri nets: Properties, analysis and applications. Proc. of the IEEE 77(4), 541–580 (1989) (1989)Google Scholar
  20. 20.
    Nevatia, R., Zhao, T., Hongeng, S.: Hierarchical language-based representation of events in video streams. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition Workshop (CVPRW 2003), vol. 4 (2003)Google Scholar
  21. 21.
    Petri, C.A.: Communication with automata. DTIC Research Report AD0630125 (1966)Google Scholar
  22. 22.
    Shoenfield, J.R.: Mathematical Logic. Addison Wesley, Reading (1967)Google Scholar
  23. 23.
    Vu, V.T., Brémond, F., Thonnat, M.: Automatic video interpretation: A novel algorithm for temporal scenario recognition. In: Proc. of the 18th Intl. Joint Conf. on Artificial Intelligence (IJCAI 2003), pp. 1295–1302 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Massimiliano Albanese
    • 1
  • Pavan Turaga
    • 1
  • Rama Chellappa
    • 1
  • Andrea Pugliese
    • 2
  • V. S. Subrahmanian
    • 1
  1. 1.Institute for Advanced Computer StudiesUniversity of MarylandCollege Park
  2. 2.DEISUniversity of CalabriaRendeItaly

Personalised recommendations