Forensic Reasoning upon Pre-Obtained Surveillance Metadata Using Uncertain Spatio-Temporal Rules and Subjective Logic

  • Seunghan Han
  • Bonjung Koo
  • Andreas Hutter
  • Walter Stechele
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 158)


This chapter presents an approach to modeling uncertain contextual rules using subjective logic for forensic visual surveillance. Unlike traditional real-time visual surveillance, forensic analysis of visual surveillance data requires mating of high level contextual cues with observed evidential metadata where both the specification of the context and the metadata suffer from uncertainties. To address this aspect, there has been work on the use of declarative logic formalisms to represent and reason about contextual knowledge, and on the use of different uncertainty handling formalisms. In such approaches, uncertainty attachment to logical rules and facts are crucial. However, there are often cases that the truth value of rule itself is also uncertain thereby, uncertainty attachment to rule itself should be rather functional. The more X then the more Y type of knowledge is one of the examples. To enable such type of rule modeling, in this chapter, we propose a reputational subjective opinion function upon logic programming, which is similar to fuzzy membership function but can also take into account uncertainty of membership value itself. Then we further adopt subjective logic’s fusion operator to accumulate the acquired opinions over time. To verify our approach, we present a preliminary experimental case study on reasoning likelihood of being a good witness that uses metadata extracted by a person tracker and evaluates the relationship between the tracked persons. The case study is further extended to demonstrate more complex forensic reasoning by considering additional contextual rules.


Visual surveillance Forensic reasoning Logic programming Subjective logic 



The work presented here was partially funded by the German Federal Ministry of Economy and Technology (BMWi) under the THESEUS project.


  1. 1.
    Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San MateoGoogle Scholar
  2. 2.
    Reiter R (1980) A logic for default reasoning. Artif Intell 13:68–93MathSciNetCrossRefGoogle Scholar
  3. 3.
    Denecker M, Kakas A (2002) Abduction in logic programming. In: Computational logic: logic programming and beyond. Lecture notes in artificial intelligence, vol 2407, pp 402–437. Springer, BerlinGoogle Scholar
  4. 4.
    Han S, Hutter A, Stechele W (2009) Toward contextual forensic retrieval for visual surveillance: challenges and an architectural approach. In: 10th international workshop on image analysis for multimedia interactive services (WIAMIS’09), London, 6–8 May 2009Google Scholar
  5. 5.
    Han S, Koo B, Hutter A, Shet V, Stechele W (2010) Subjective logic based hybrid approach to conditional evidence fusion for forensic visual surveillance. In: 7th IEEE international conference on advanced video and signal based surveillance (AVSS’10), Boston, 29 Aug–1 Sept 2010Google Scholar
  6. 6.
    Han S, Koo B, Stechele W (2010) Subjective logic based approach to modeling default reasoning for visual surveillance. In: 4th IEEE international conference on semantic computing (ICSC’10), Pittsburgh, 22–24 Sept 2010Google Scholar
  7. 7.
    Jøsang A (2001) A logic for uncertain probabilities. Int J Uncertain Fuzz Knowl-Based Syst 9:279–311Google Scholar
  8. 8.
    Bremond F, Thonnat M (1996) A context representation for surveillance systems. In: ECCV worshop on conceptual descriptions from images, Cambridge, Apr 1996Google Scholar
  9. 9.
    Akdemir U, Turaga P, Chellappa R (2008) An ontology based approach for activity recognition from video. In: ACM conference on multimedia (ACM-MM’08), Oct 2008Google Scholar
  10. 10.
    Jianbing M, Weiru L, Paul M, Weiqi Y (2009) Event composition with imperfect information for bus surveillance. In: 6th IEEE international conference on advanced video and signal based surveillance (AVSS’09), Genoa, 2–4 Sept 2009Google Scholar
  11. 11.
    Shet V, Harwood D, Davis L (2005) VidMAP: video monitoring of activity with prolog. In: 2nd IEEE international conference on advanced video and signal based surveillance (AVSS’05), Como, 15–16 Sept 2005Google Scholar
  12. 12.
    Shet V, Neumann J, Ramesh V, Davis L (2007) Bilattice-based logical reasoning for human detection. In: IEEE international conference on computer vision and pattern recognition (CVPR’07), Minneapolis, 18–23 June 2007Google Scholar
  13. 13.
    Anderson D, Luke RH, Keller JM, Skubic M (2007) Modeling human activity from voxel person using fuzzy logic. IEEE Trans Fuzzy Syst 17(1):39–49Google Scholar
  14. 14.
    Hongeng S, Nevatia R, Bremond F (2004) Video-based event recognition: activity representation and probabilistic recognition methods. Comput Vis Image Understand 96(2):129–162CrossRefGoogle Scholar
  15. 15.
    Feryhough J, Cohn AG, Hogg DC (1998) Building qualitative event models automatically from visual input. In: 6th IEEE international conference on computer vision (ICCV’98), pp 350–355Google Scholar
  16. 16.
    Makris D, Ellis T (2002) Spatial and probabilistic modelling of pedestrian behaviour. In: 13th British machine vision conference (BMVC’02), Cardiff, 2–5 Sept 2002Google Scholar
  17. 17.
    Cupillard F, Bremond F, Thonnat M (2003) Behavior recognition for individuals, groups of people, and crowd. In: IEEE seminar intelligent distributed surveillance systems, London, Mar 2003Google Scholar
  18. 18.
    Makris D, Ellis T (2005) Learning semantic scene models from observing activity in visual surveillance. IEEE Trans Syst Man Cybernet Part B 35(3):397–408Google Scholar
  19. 19.
    Ginsberg ML (1988) Multivalued logics: a uniform approach to inference in artificial intelligence. Comput Intell 4(3):256–316Google Scholar
  20. 20.
    Shafer G (1976) A mathematical theory of evidence. Princeton University Press, PrincetonGoogle Scholar
  21. 21.
    Zadeh LA (1973) Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans Syst Man Cybernet (SMC)(3):28–44Google Scholar
  22. 22.
    Jøsang A (1997) Artificial reasoning with subjective logic. In: 2nd Australian workshop on commonsense reasoning, PerthGoogle Scholar
  23. 23.
    NASA (1993) Basic programming guide. NASA JSC-25012Google Scholar
  24. 24.
    Mizumoto M, Zimmermann HJ (1982) Comparison of fuzzy reasoning methods. Fuzzy Sets Syst 8:253–283MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Gottwald S, Hajek P (2005) Triangular norm based mathematical fuzzy logic. In: Klement EP, Mesiar R (eds) Logical, algebraic, analytic and probabilistic aspects of triangular norms, pp 275–300. Elsevier, AmsterdamGoogle Scholar
  26. 26.
    Zadeh LA (2008) Is there a need for fuzzy logic? Inf Sci 178:2751–2779Google Scholar
  27. 27.
    Jøsang A, Marsh S, Pope S (2006) Exploring different types of trust propagation. In: 4th international conference on trust managementGoogle Scholar
  28. 28.
    i-lids dataset.:
  29. 29.
    Jøsang A, McAnally D (2004) Multiplication and comultiplication of beliefs. Int J Approx Reason 142:19–51Google Scholar
  30. 30.
    Jøsang A (2006) The consensus operator for combining beliefs. Artif Intell J 38(1):157–170Google Scholar
  31. 31.
    Hakeem A, Shah M (2004) Ontology and taxonomy collaborated framework for meeting classification. In: 17th international conference on pattern recognition (ICPR’04), WashingtonGoogle Scholar
  32. 32.
    Jøsang A (2008) Conditional reasoning with subjective logic. J Multiple Valued Logic Soft Comput 15(1):5–38Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Seunghan Han
    • 3
    • 1
  • Bonjung Koo
    • 1
    • 2
  • Andreas Hutter
    • 1
  • Walter Stechele
    • 3
  1. 1.Siemens AG, Corporate TechnologyMunichGermany
  2. 2.Department of Computer ScienceSogang UniversitySeoulKorea
  3. 3.Institute for Integrated SystemsTechnische Universität MünchenMunichGermany

Personalised recommendations