Ontology-Based Realtime Activity Monitoring Using Beam Search

  • Wilfried Bohlken
  • Bernd Neumann
  • Lothar Hotz
  • Patrick Koopmann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6962)


In this contribution we present a realtime activity monitoring system, called SCENIOR (SCEne Interpretation with Ontology-based Rules) with several innovative features. Activity concepts are defined in an ontology using OWL, extended by SWRL rules for the temporal structure, and are automatically transformed into a high-level scene interpretation system based on JESS rules. Interpretation goals are transformed into hierarchical hypotheses structures associated with constraints and embedded in a probabilistic scene model. The incremental interpretation process is organised as a Beam Search with multiple parallel interpretation threads. At each step, a context-dependent probabilistic rating is computed for each partial interpretation reflecting the probability of that interpretation to reach completion. Low-rated threads are discarded depending on the beam width. Fully instantiated hypotheses may be used as input for higher-level hypotheses, thus realising a doubly hierarchical recognition process. Missing evidence may be "hallucinated" depending on the context. The system has been evaluated with real-life data of aircraft service activities.


Partial Interpretation Markov Logic Network Primitive Event Scene Interpretation SWRL Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fusier, F., Valentin, V., Brémond, F., Thonnat, M., Borg, M., Thirde, D., Ferryman, J.: Video Understanding for Complex Activity Recognition. Machine Vision and Applications 18(3), 167–188 (2007)CrossRefzbMATHGoogle Scholar
  2. 2.
    Cohn, A.G., Magee, D., Galata, A., Hogg, D., Hazarika, S.: Towards an architecture for cognitive vision using qualitative spatio-temporal representations and abduction. In: Freksa, C., Brauer, W., Habel, C., Wender, K.F. (eds.) Spatial Cognition III. LNCS (LNAI), vol. 2685, pp. 232–248. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  3. 3.
    Shanahan, M.: Perception as abduction: Turning sensor data into meaningful representation. Cognitive Science 29, 103–134 (2005)CrossRefGoogle Scholar
  4. 4.
    Moeller, R., Neumann, B.: Ontology-Based Reasoning Techniques for Multimedia Interpretation and Retrieval. In: Kompatsiaris, Y., Hobson, P. (eds.) Semantic Multimedia and Ontologies: Theory and Applications, pp. 55–98. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Morariu, V.I., Davis, L.S.: Multi-agent event recognition in structured scenarios. In: CVPR 2011, IEEE Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  6. 6.
    Neumann, B.: Bayesian Compositional Hierarchies - A Probabilistic Structure for Scene Interpretation. TR FBI-HH-B-282/08, Univ. of Hamburg, Dep. of Informatics (2008)Google Scholar
  7. 7.
    Neumann, B., Möller, R.: On Scene Interpretation with Description Logics. In: Christensen, H.I., Nagel, H.-H. (eds.) Cognitive Vision Systems. LNCS, vol. 3948, pp. 247–275. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Norvig, P.: Paradigms of Artificial Intelligence. Morgan Kaufmann, San Francisco (1992)Google Scholar
  9. 9.
    Bohlken, W., Neumann, B.: Generation of Rules from Ontologies for High-Level Scene Interpretation. In: Governatori, G., Hall, J., Paschke, A. (eds.) RuleML 2009. LNCS, vol. 5858, pp. 93–107. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Wilfried Bohlken
    • 1
  • Bernd Neumann
    • 1
  • Lothar Hotz
    • 1
  • Patrick Koopmann
    • 1
  1. 1.FB InformatikUniversität HamburgGermany

Personalised recommendations