Simulated Virtual Crowds Coupled with Camera-Tracked Humans

  • Ivan Rivalcoba
  • Oriam De Gyves
  • Isaac Rudomin
  • Nuria Pelechano
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 550)


Our objective with this paper is to show how we can couple a group of real people and a simulated crowd of virtual humans. We attach group behaviors to the simulated humans to get a plausible reaction to real people. We use a two stage system: in the first stage, a group of people are segmented from a live video, then a human detector algorithm extracts the positions of the people in the video, which are finally used to feed the second stage, the simulation system. The positions obtained by this process allow the second module to render the real humans as avatars in the scene, while the behavior of additional virtual humans is determined by using a simulation based on a social forces model. Developing the method required three specific contributions: a GPU implementation of the codebook algorithm that includes an auxiliary codebook to improve the background subtraction against illumination changes; the use of semantic local binary patterns as a human descriptor; the parallelization of a social forces model, in which we solve a case of agents merging with each other. The experimental results show how a large virtual crowd reacts to over a dozen humans in a real environment.


  1. 1.
    Ahonen, T.: Face description with local binary patterns: application to face recognition. Pattern Anal. Mach. Intell. 28(12), 41–2037 (2006). Scholar
  2. 2.
    Banerjee, P., Sengupta, S.: Human motion detection and tracking for video surveillance. In: Proceedings of the National Conference of Tracking and Video Surveillance Activity Analysis, pp. 88–92 (2008)Google Scholar
  3. 3.
    Van den Berg, J., Guy, S.J., Lin, M., Manocha, D.: Reciprocal n-body collision avoidance. Robot. Res. 70, 3–19 (2011). Scholar
  4. 4.
    Van den Berg, J., Manocha, D.: Reciprocal velocity obstacles for real-time multi-agent navigation. In: 2008 IEEE International Conference on Robotics and Automation, pp. 1928–1935. IEEE, May 2008.
  5. 5.
    Bhuvaneswari, K., Rauf, H.A.: Edgelet based human detection and tracking by combined segmentation and soft decision. In: Control, Automation, Communication and Energy Conservation, 4–9 June 2009.
  6. 6.
    Bleiweiss, A.: Multi agent navigation on GPU. White paper, GDC (2009).
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893 (2005).
  8. 8.
    De Gyves, O., Toledo, L., Rudomín, I.: Comportamientos en simulación de multitudes : revisión del estado del arte. Res. Comput. Sci. 62, 319–334 (2013). Special Issue: Avances en Inteligencia ArtificialGoogle Scholar
  9. 9.
    Helbing, D., Farkas, I., Vicsek, T.: Simulating dynamical features of escape panic. Nature 407(6803), 90–487 (2000). Scholar
  10. 10.
    Helbing, D., Molnár, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282–4286 (1995). Scholar
  11. 11.
    Huang, T.: Discriminative local binary patterns for human detection in personal album. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition. pp. 1–8. IEEE, June 2008.
  12. 12.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Real-time foregroundbackground segmentation using codebook model. Real-Time Imaging 11(3), 172–185 (2005)CrossRefGoogle Scholar
  13. 13.
    Lengvenis, P., Simutis, R., Vaitkus, V., Maskeliunas, R.: Application of computer vision systems for passenger counting in public transport. Electron. Electr. Eng. 19(3), 69–72 (2013). Scholar
  14. 14.
    Li, M., Zhang, Z., Huang, K., Tan, T.: Rapid and robust human detection and tracking based on omega-shape features. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2545–2548. IEEE, November 2009.
  15. 15.
    Li, T.Y., Wen Lin, J., Liu, Y.L., Ming Hsu, C.: Interactively directing virtual crowds in a virtual environment. Conf. Artif. Real Telexistence vol. 10 (2002).
  16. 16.
    Millan, E., Hernandez, B., Rudomin, I.: Large crowds of autonomous animated characters using fragment shaders and level of detail. In: Engel, W. (ed.) ShaderX5: Advanced Rendering Techniques, chap. Beyond Pix, pp. 501–510. Charles River Media (2006).
  17. 17.
    Moussaïd, M., Perozo, N., Garnier, S., Helbing, D., Theraulaz, G.: The walking behaviour of pedestrian social groups and its impact on crowd dynamics. PloS one 5(4), e10047 (2010)CrossRefGoogle Scholar
  18. 18.
    Mukherjee, S., Das, K.: Omega model for human detection and counting for application in smart surveillance system. Int. J. Adv. Comput. Sci. Appl. 4(2), 167–172 (2013). arXiv preprint arXiv:1303.0633Google Scholar
  19. 19.
    Ojala, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002). Scholar
  20. 20.
    Ozturk, O., Yamasaki, T., Aizawa, K.: Tracking of humans and estimation of body/head orientation from top-view single camera for visual focus of attention analysis. In: Computer Vision Workshops (ICCV Workshops), pp. 1020–1027 September 2009.
  21. 21.
    Pelechano, N., Stocker, C.: Being a part of the crowd: towards validating VR crowds using presence. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pp. 12–16 (2008).
  22. 22.
    Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. ACM SIGGRAPH Comput. Graph. 21(4), 25–34 (1987). Scholar
  23. 23.
    Rivalcoba, I.J., Rudomin, I.: Segmentación de peatones a partir de vistas aéreas. Res. Comput. Sci. 62, 129–230 (2013)Google Scholar
  24. 24.
    Tuzel, O., Porikli, F., Meer, P.: Human detection via classification on Riemannian manifolds. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 June 2007.
  25. 25.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I-511–I-518 (2001).
  26. 26.
    Viola, P., Jones, M., Snow, D.: Detecting pedestrians using patterns of motion and appearance. Int. Conf. Comput. Vision 63(2), 153–161 (2003). Scholar
  27. 27.
    Wang, Y., Dubey, R., Magnenat-Thalmann, N., Thalmann, D.: An immersive multi-agent system for interactive applications. Vis. Comput. 29(5), 323–332 (2012). Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is distributed under the terms of the Creative Commons Attribution Noncommercial License, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  • Ivan Rivalcoba
    • 1
  • Oriam De Gyves
    • 1
  • Isaac Rudomin
    • 2
  • Nuria Pelechano
    • 3
  1. 1.Department of Computer ScienceTecnológico de MonterreyMexico CityMexico
  2. 2.Computer SciencesBarcelona Supercomputing CenterBarcelonaSpain
  3. 3.Llenguatges I Sistemes InformàticsUniversitat Politècnica de CatalunyaBarcelonaSpain

Personalised recommendations