Skip to main content
SpringerLink
Log in

Augmentation of virtual agents in real crowd videos

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Augmenting virtual agents in real crowd videos is an important task for different applications from simulations of social environments to modeling abnormalities in crowd behavior. We propose a framework for this task, namely for augmenting virtual agents in real crowd videos. We utilize pedestrian detection and tracking algorithms to automatically locate the pedestrians in video frames and project them into our simulated environment, where the navigable area of the simulated environment is available as a navigation mesh. We represent the real pedestrians in the video as simple three-dimensional (3D) models in our simulation environment. 3D models representing real agents and the augmented virtual agents are simulated using local path planning coupled with a collision avoidance algorithm. The virtual agents augmented into the real video move plausibly without colliding with static and dynamic obstacles, including other virtual agents and the real pedestrians.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Lerner, A., Chrysanthou, Y., Lischinski, D.: Crowds by example. Comput. Gr. Forum 26(3), 655–664 (2007)

    Article  Google Scholar 

  2. Charalambous, P., Karamouzas, I., Guy, S.J., Chrysanthou, Y.: A data-driven framework for visual crowd analysis. Comput. Gr. Forum 33(7), 41–50 (2014)

    Article  Google Scholar 

  3. Musse, S.R., Jung, C.R., Jacques, J., Braun, A.: Using computer vision to simulate the motion of virtual agents. Comput. Anim. Virtual Worlds 18(2), 83–93 (2007)

    Article  Google Scholar 

  4. Kim, S., Bera, A., Best, A., Chabra, R., Manocha, D.: Interactive and adaptive data-driven crowd simulation. In: Proceedings of IEEE Virtual Reality, ser. VR’16, pp. 29–38 (2016)

  5. Narahara, T., Kobayashi, Y.: Crowd mapper: projection-based interactive pedestrian agents for collective design in architecture. In: Proceedings of the eCAADe 33rd Annual Conference. Education and Research in Computer Aided Architectural Design in Europe (eCAADe), pp. 191–200 (2015)

  6. Bulbul, A., Dahyot, R.: Populating virtual cities using social media. Comput. Anim. Virtual Worlds 28(5), e1742 (2017)

    Article  Google Scholar 

  7. Turkay, C., Koc, E., Balcisoy, S.: Integrating information theory in agent-based crowd simulation behavior models. Comput. J. 54(11), 1800–1820 (2011)

    Article  Google Scholar 

  8. Thalmann, N.M., Thalmann, D.: Animating virtual actors in real environments. Multimed. Syst. 5(2), 113–125 (1997)

    Article  Google Scholar 

  9. Zheng, F., Li, H.: ARCrowd-a tangible interface for interactive crowd simulation. In: Proceedings of the 16th International Conference on Intelligent User Interfaces. ACM, pp. 427–430 (2011)

  10. Obaid, M., Damian, I., Kistler, F., Endrass, B., Wagner, J., André, E.: Cultural behaviors of virtual agents in an augmented reality environment. In: Proceedings of the International Conference on Intelligent Virtual Agents. Springer, pp. 412–418 (2012)

  11. Fernández, C., Baiget, P., Roca, F.X., Gonzàlez, J.: Augmenting video surveillance footage with virtual agents for incremental event evaluation. Pattern Recognit. Lett. 32(6), 878–889 (2011)

    Article  Google Scholar 

  12. Baiget, P., Fernández, C., Roca, X., Gonzàlez, J.: Generation of augmented video sequences combining behavioral animation and multi-object tracking. Comput. Anim. Virtual Worlds 20(4), 473–489 (2009)

    Article  Google Scholar 

  13. Bera, A., Randhavane, T., Prinja, R., Manocha, D.: Sociosense: robot navigation amongst pedestrians with social and psychological constraints. arXiv:1706.01102 (2017)

  14. Van den Berg, J., Lin, M., Manocha, D.: Reciprocal velocity obstacles for real-time multi-agent navigation. In: Proceedings of the IEEE International Conference on Robotics and Automation, ser. ICRA’08. IEEE, pp. 1928–1935 (2008)

  15. Zhang, G., Qin, X., An, X., Chen, W., Bao, H.: As-consistent-as-possible compositing of virtual objects and video sequences. Comput. Anim. Virtual Worlds 17(3–4), 305–314 (2006)

    Article  Google Scholar 

  16. Unity Technologies, Inc., Unity manual: navigation and pathfinding. [Online]. https://docs.unity3d.com/Manual/Navigation.html

  17. Pelechano, N., Fuentes, C.: Hierarchical path-finding for navigation meshes (HNA*). Comput. Gr. 59, 68–78 (2016)

    Article  Google Scholar 

  18. Olivier, A.-H., Bruneau, J., Kulpa, R., Pettré, J.: Walking with virtual people: evaluation of locomotion interfaces in dynamic environments. IEEE Trans. Vis. Comput. Gr. 24(7), 2251–2263 (2018)

    Article  Google Scholar 

  19. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, ser. CVPR’05, IEEE, vol. 1, pp. 886–893 (2005)

  20. Zivkovic, Z., van der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit. Lett. 27(7), 773–780 (2006)

    Article  Google Scholar 

  21. Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: MOTChallenge 2015: towards a benchmark for multi-target tracking. arXiv:1504.01942 [cs] (2015)

  22. Takahashi, K.: Shadowdrawer (2015). [Online]. Available: https://github.com/keijiro/ShadowDrawer

  23. García-Martín, Á., Sánchez-Matilla, R., Martínez, J.M.: Hierarchical detection of persons in groups. Signal Image Video Process. 11(7), 1181–1188 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Bilkent, 06800, Ankara, Turkey

    Yalım Doğan, Serkan Demirci, Uğur Güdükbay & Hamdi Dibeklioğlu

Authors
  1. Yalım Doğan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Serkan Demirci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Uğur Güdükbay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hamdi Dibeklioğlu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Uğur Güdükbay.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 5533 KB)

Supplementary material 2 (mp4 107983 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doğan, Y., Demirci, S., Güdükbay, U. et al. Augmentation of virtual agents in real crowd videos. SIViP 13, 643–650 (2019). https://doi.org/10.1007/s11760-018-1392-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1392-8

Keywords

Search

Navigation