Event Based Switched Dynamic Bayesian Networks for Autonomous Cognitive Crowd Monitoring

  • Simone Chiappino
  • Lucio Marcenaro
  • Pietro Morerio
  • Carlo Regazzoni
Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 6)


Human behavior analysis is one of the most important applications in Intelligent Video Surveillance (IVS) field. In most recent systems addressed by research, automatic support to the human decisions based on object detection, tracking and situation assessment tools is integrated as a part of a complete cognitive artificial process including security maintenance procedures actions that are in the scope of the system. In such cases an IVS needs to represent complex situations that describe alternative possible real time interactions between the dynamic observed situation and operators’ actions. To obtain such knowledge, particular types of Event based Dynamic Bayesian Networks E-DBNs are here proposed that can switch among alternative Bayesian filtering and control lower level modules to capture adaptive reactions of human operators. It is shown that after the off line learning phase Switched E-DBNs can be used to represent and anticipate possible operators’ actions within the IVS. In this sense acquired knowledge can be used for either fully autonomous security preserving systems or for training of new operators. Results are shown by considering a crowd monitoring application in a critical infrastructure. A system is presented where a Cognitive Node (CN) embedding in a structured way Switched E-DBN knowledge can interact with an active visual simulator of crowd situations. It is also shown that outputs from such a simulator can be easily compared with video signals coming from real cameras and processed by typical Bayesian tracking methods.


Crowding Dynamic Bayesian Networks Cognitive Systems Bio-inspired learning 


  1. 1.
    Remagnino, P., Velastin, S.A., Foresti, G.L., Trivedi, M.: Novel concepts and challenges for the next generation of video surveillance systems. Mach. Vis. Appl. 18(3), 135–137 (2007)CrossRefGoogle Scholar
  2. 2.
    Trivedi, M., et al.: Intelligent environments and active camera networks. In: Proceedings of the IEEE International Conference on System, Man and Cybernetics, vol. 2, pp. 804–809 (2000)Google Scholar
  3. 3.
    Lipton, A., Heartwell, C., Haering, N., Madden, D.: Automated video protection, monitoring & detection. IEEE Aerosp. Electron. Syst. Mag. 18(5), 3–18 (2003)CrossRefGoogle Scholar
  4. 4.
    Trivedi, M.M., et al.: Looking-in and looking-out of a vehicle: computer-vision-based enhanced vehicle safety. IEEE Trans. Intell. Transp. Syst. 8(1), 108–120 (2007)Google Scholar
  5. 5.
    Damasio, A.R.: The Feeling of What Happens-Body, Emotion and the Making of Consciousness. Harvest Books, New York (2000)Google Scholar
  6. 6.
    Valera, M., Velastin, S.: Intelligent distributed surveillance systems: a review. IEEE Proc. Vis. Image Signal Process. 52(2), 192–204 (2005)CrossRefGoogle Scholar
  7. 7.
    Foresti, G.L., Regazzoni, C.S., Varshney, P.K.: Multisensor Surveillance Systems: The Fusion Perspective. Kluwer Academic, Boston (2003)CrossRefGoogle Scholar
  8. 8.
    Collins, R., Lipton, A., Fujiyoshi, H., Kanade, T.: Algorithms for cooperative multisensory surveillance. Proc. IEEE 89(10), 1456–1477 (2001)CrossRefGoogle Scholar
  9. 9.
    Smith, D., Singh, S.: Approaches to multisensor data fusion in target tracking: a survey. IEEE Trans. Knowl. Data Eng. 18(12), 1696–1710 (2006)CrossRefGoogle Scholar
  10. 10.
    Prati, A., et al.: An integrated multi-modal sensor network for video surveillance. In: Proceedings of the Third ACM International Workshop on Video Surveillance & Sensor Networks (2005)Google Scholar
  11. 11.
    Chang, B.R., Tsai, H.F., Young, C.-P.: Intelligent data fusion system for predicting vehicle collision warning using vision/gps sensing. Expert. Syst. Appl. 37(3), 2439–2450 (2010)CrossRefGoogle Scholar
  12. 12.
    Wu, S., Decker, S., Chang, P., Camus, T., Eledath, J.: Collision sensing by stereo vision and radar sensor fusion. IEEE Trans. Intell. Transp. Syst. 10(4), 606–614 (2009)CrossRefGoogle Scholar
  13. 13.
    Zhan, B., Monekosso, D.N., Remagnino, P., Velastin, S.A., Xu, L.-Q.: Crowd analysis: a survey. Mach. Vis. Appl. 19, 345–357 (2008)CrossRefMATHGoogle Scholar
  14. 14.
    Loscos, C., Marchal, D., Meyer, A.: Intuitive crowd behavior in dense urban environments using local laws. In: Proceedings of the Theory and Practice of Computer Graphics, pp. 122–129 (2003)Google Scholar
  15. 15.
    Liu, B., Liu, Z., Hong, Y.: A simulation based on emotions model for virtual human crowds. In: Fifth International Conference on Image and Graphics ICIG’09, pp. 836–840 (2009)Google Scholar
  16. 16.
    Handford, D., Rogers, A.: Modelling driver interdependent behavior in agent-based traffic simulations for disaster management. In: The Ninth International Conference on Practical Applications of Agents and Multi-Agent Systems, pp. 163–172 (2011)Google Scholar
  17. 17.
    Davies, A.C., Yin, J.H., Velastin, S.A.: Crowd monitoring using image processing. Electron. Commun. Eng. J. 7, 37–47 (1995)CrossRefGoogle Scholar
  18. 18.
    Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.P.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19, 780–785 (1997)CrossRefGoogle Scholar
  19. 19.
    Haritaoglu, I., Harwood, D., David, L.S.: W4: real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell. 22, 809–830 (2000)CrossRefGoogle Scholar
  20. 20.
    Marana, A.N., Velastin, S.A., Costa, L.F., Lotufo, R.A.: Automatic estimation of crowd density using texture. Safety Sci. 28(3), pp. 165–175 (1998)CrossRefGoogle Scholar
  21. 21.
    Zhao, T., Nevatia, R.: Bayesian human segmentation in crowded situations. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, p. 459 (2003)Google Scholar
  22. 22.
    Andrade, E., Blunsden, S., Fisher, R.: Hidden markov models for optical flow analysis in crowds. In: 18th International Conference on Pattern Recognition.(ICPR), vol. 1, pp. 460–463 (2006)Google Scholar
  23. 23.
    Benabbas, Y., Ihaddadene, N., Djeraba, C.: Motion pattern extraction and event detection for automatic visual surveillance. EURASIP J. Image Video Process. 2011, 15 (2011)CrossRefGoogle Scholar
  24. 24.
    Rahmalan, H., Nixon, M., Carter, J.: On crowd density estimation for surveillance. In: The Institution of Engineering and Technology Conference on Crime and Security, pp. 540–545 (2006)Google Scholar
  25. 25.
    Cupillard, F., Avanzi, A., Bremond, F., Thonnat, M.: Video understanding for metro surveillance. In: IEEE International Conference on Networking, Sensing and Control, vol. 1, pp. 186–191 (2004)Google Scholar
  26. 26.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE conference on computer vision and pattern recognition CVPR 2009, pp. 935–942 (2009)Google Scholar
  27. 27.
    Pellegrini, S., Ess, A., Schindler, K., Gool, L. van.: You’ll never walk alone: modeling social behavior for multi-target tracking. In: International Conference on Computer Vision, Miami, 20–26 June 2009Google Scholar
  28. 28.
    Luber, M., Stork, J.A., Tipaldi, G.D., Arras, K.O.: People tracking with human motion predictions from social forces. In: Proceedings of the International Conference on Robotics & Automation (ICRA), Anchorage (2010)Google Scholar
  29. 29.
    Moore, B.E., Ali, S., Mehran, R., Shah, M.: Visual crowd surveillance through a hydrodynamics lens. Commun. ACM 54(12), 64–73 (2011)CrossRefGoogle Scholar
  30. 30.
    Dore, A., Regazzoni, C.S.: Bayesian bio-inspired model for learning interactive trajectories. In: Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, Genoa, Italy, 2–4 Sept 2009Google Scholar
  31. 31.
    Dore, A., Cattoni, A., Regazzoni, C.: Interaction modeling and prediction in smart spaces: a bio-inspired approach based on autobiographical memory. In: IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans (2010)Google Scholar
  32. 32.
    Kohonen, T.: The self-organizing map. Proc. IEEE 78(9), 1464–1480 (1990)CrossRefGoogle Scholar
  33. 33.
    Dore, A., Soto, M., Regazzoni, C.S.: Bayesian tracking for video analytics. Signal Process. Mag. IEEE. 27(5), pp.46–55, (2010) doi:  10.1109/MSP.2010.937395 Google Scholar
  34. 34.
    Patnaik, D., Laxman, S., Ramakrishnan, N.: Discovering excitatory networks from discrete event streams with applications to neuronal spike train analysis. In: ICDM, pp. 407–416 (2009)Google Scholar
  35. 35.
    Oliver, N.M., Rosario, B., Pentland, A.P.: A bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)CrossRefGoogle Scholar
  36. 36.
    Pan, W, Dong, W, Cebrian, M., Kim, T., Fowler, J.H., Pentland, A.S.: Modeling dynamical influence in human interaction: using data to make better inferences about influence within social systems. IEEE Signal Process. Mag. 29(2), 77–86 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Simone Chiappino
    • 1
  • Lucio Marcenaro
    • 1
  • Pietro Morerio
    • 1
  • Carlo Regazzoni
    • 1
  1. 1.Signal Processing and Telecommunications Group, Department of Naval, Electrical, Electronic and Telecommunication EngineeringUniversity of GenoaGenoaItaly

Personalised recommendations