Skip to main content
SpringerLink
Log in

Real-time identification of pedestrian meeting and split events from surveillance videos using motion similarity and its applications

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

A real-time system to automatically identify pedestrian meeting events from surveillance videos is proposed. The system consists of three components: a pedestrian detection and tracking module, a pedestrian group identification module and a pedestrian group record. A three-level blob filter is used to improve the accuracy of pedestrian detection in the pedestrian detection and tracking module. Our previous work, the non-recursive motion similarity clustering algorithm is used as the pedestrian group identification module. Groups are detected within a time period of 0.02 ms (for four pedestrians in the scene) to 0.05 ms (for 32 pedestrians in the scene) of their occurrences in the video, using an Intel I7 processor-based machine. The pedestrian groups identified by this algorithm are stored in pedestrian group records, which are used subsequently to identify pedestrian meeting events. Visualizations were created to highlight the pedestrian groups, their history of group membership and the spatial distribution. With these visualizations, the enforcement agencies no longer need to browse through entire video archives for investigation purposes. We implemented the system to monitor several locations simultaneously in residential halls at the National University of Singapore. Our system was able to handle successfully 18 digital 640 \(\times\) 480 pixel video streams at 25 fps on a moderately loaded Ethernet, monitoring a maximum of 30 pedestrians and detecting 83 % of the meeting and split events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Alfredo, P., Marco, M., Ferone, A.: A real-time streaming server in the RTLinux environment using VideoLanClient. J Real Time Image Process 6, 247–256 (2011)

    Article  Google Scholar 

  2. Barros, J., French, J., Martin, W., Kelly, P., Cannon, M.: Using the triangle inequality to reduce the number of comparisons required for similarity-based retrieval. In: Proceedings of SPIE conf on storage and retrieval for image and video databases, 392–403 (1996)

  3. Berry, C.: The kappa statistic. J Am Med Assoc 268(18), 2513–2514 (1992)

    Article  Google Scholar 

  4. Bing-bing W., Zhi-xin C., Jia W., Liquan Z.: Pedestrian detection based on the combination of HOG and background subtraction method. In: Proceedings of Intl Conf on transportation, mechanical, and electrical engineering (TMEE), 527–531 (2011)

  5. Canny, J.: A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6), 679–698 (1986)

    Article  Google Scholar 

  6. CAVIAR research pedestrian data set. http://www-prima.inrialpes.fr/PETS04/caviar_data.html (2004). Accessed 4 Feb 2016

  7. Chandran A.K., Ai Poh L., Vadakkepat P.: Identifying social groups in pedestrian crowd videos. In: Proceedings of Intl Conf on advances in pattern recognition (ICAPR), 1–6 (2015)

  8. Chandran A.K.: Pedestrian-groups data sets. http://arunkumarchandran.net/home-page/data-sets (2016). Accessed 28 Mar 2016 (data set available after signing a declaration form)

  9. Cupillard F., Bremond F., Thonnat M.: Tracking groups of people for video surveillance. In: Proceedings of European workshop on advanced video-based surveillance system, 89–100 (2001)

  10. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 1, 886–893 (2005)

    Google Scholar 

  11. David, M.W.P.: Evaluation: from precision, recall and F-factor to ROC. J Mach Learn Technol Inf Mark Correl 2(1), 3763 (2007)

    Google Scholar 

  12. ETH Research pedestrian data set. http://www.vision.ee.ethz.ch/datasets/index.en.html (2009). Accessed 4 Feb 2016

  13. Forczmaski, P., Kukharev, G.: Comparative analysis of simple facial features extractors. J Real Time Image Process 1, 239–255 (2007)

    Article  Google Scholar 

  14. French A., Naeem A., Dryden I., Pridmore T.: Using social effects to guide tracking in complex scenes. In: Proceedings of IEEE Conf. on advanced video and signal based surveillance, 212–217 (2007)

  15. Gatica-Perez, D.: Automatic nonverbal analysis of social interaction in small groups: a review. Image Vis Comput Spec Issue Hum Behav 27, 1775–1787 (2009)

    Article  Google Scholar 

  16. Ge, W., Collins, R.T.: Vision-based analysis of small groups in pedestrian crowds. IEEE Trans Pattern Anal Mach Intell 34(5), 1003–1016 (2012)

    Article  Google Scholar 

  17. Guler P., Emeksiz D., Temizel A., Teke M., Temizel TT.: Real-time multi-camera video analytics system on GPU. J Real Time Image Process, 1–16 (2013)

  18. Hall, E.T.: A system for the notation of proxemic behaviour. Am Anthropol 65, 1003–1026 (1963)

    Article  Google Scholar 

  19. Haritaoglu I., Flickner M.: Detection and tracking of shopping groups in stores. In: Proceedings of IEEE conf. on computer vision and pattern recognition, 431–438 (2001)

  20. Hoogs A., Bush S., Brooksby G., Perera A., Dausch M., Krahnstoever N.: Detecting semantic group activities using relational clustering. In: Proceedings of IEEE workshop on motion and video computing, 1–8 (2008)

  21. Information on people tracking using Kalman filters. http://www.mathworks.com/help/vision/examples/people-tracking.html (2003). Accessed 4 Feb 2016

  22. Ishii, I., Ichida, T., Gu, Q., Takaki, T.: 500-fps face tracking system. J Real Time Image Process 8, 379–388 (2013)

    Article  Google Scholar 

  23. Jacques-Junior, J., Braun, A., Soldera, J., Musse, S., Jung, C.: Understanding people motion in video sequences using Voronoi diagrams. J Pattern Anal Appl 10, 321–332 (2007)

    Article  Google Scholar 

  24. Kuhn, H.W.: The Hungarian method for the assignment problem. Nav Res Logist Q 2, 83–97 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  25. Lau, B., Arras, K., Burgard, W.: Multi-model hypothesis group tracking and group size estimation. Intl J Soc Robot 2(1), 19–30 (2010)

    Article  Google Scholar 

  26. McPhail, C., Wohlstein, R.: Using film to analyze pedestrian behavior. Sociol Methods Res 10, 347–375 (1982)

    Article  Google Scholar 

  27. McPhail C.: The myth of the madding crowd, book on crowd behavior, 34–39 (1991)

  28. Morris, B.T., Trivedi, M.M.: Learning, modeling, and classification of vehicle track patterns from live video. IEEE Trans Intell Transp Syst 9(3), 425–437 (2008)

    Article  Google Scholar 

  29. Naturel X., Odobez J.: Detecting queues at vending machines—a statistical layered approach. In: Proceedings of Intl Conf. on pattern recognition, 1–4 (2008)

  30. Piccardi, M.: Background subtraction techniques—a review. IEEE Int Conf Syst Man Cybern 4, 3099–3104 (2004)

    Google Scholar 

  31. Reid, D.: An algorithm for tracking multiple targets. IEEE Trans Autom control 24(6), 843–854 (1979)

    Article  Google Scholar 

  32. Report on internet bandwidth usage at the National University of Singapore. http://www.nus.edu.sg/comcen/gethelp/guide/itcare/bandwidth.htm (2015). Accessed 4 Feb 2016

  33. Sochman J., Hogg D.C: Who knows who-inverting the social force model for finding groups. In: Proceedings of IEEE Conf. on computer vision workshops, 830–837 (2011)

  34. Stauffer, C., Grimson, W.: Learning patterns of activity using real time tracking. IEEE Trans Pattern Anal Mach Intell 22(8), 747767 (2000)

    Article  Google Scholar 

  35. Vlachos M., Kollios G., Gunopulos D.: Discovering similar multidimensional trajectories. In: Proceedings of IEEE Conf. on data engineering, 673–684 (2002)

  36. Wirz M., Kjrgaard M.B., Feese S., Schlpfer P., Roggen D., Trster G.: Towards an online detection of pedestrian flocks in urban canyons by smoothed spatio-temporal clustering of GPS. Association for Computing Machinery- Location Based Sensory Networks, 17–24 (2011)

Download references

Acknowledgments

We would like to thank the NUS Sheares Hall, NUS Libraries for the video footage and the NUS Ambient Intelligence (AMI) lab for their support. We would like to acknowledge that Tables 3, 4 and 6, Figs. 7 and 10 are from one of our previous conference proceedings publication (doi:10.1109/ICAPR.2015.7050677) which is cited in this paper. They are used in this paper to explain the RPMVIS system.

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore

    Arun Kumar Chandran, Loh Ai Poh & Prahlad Vadakkepat

Authors
  1. Arun Kumar Chandran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Loh Ai Poh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Prahlad Vadakkepat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arun Kumar Chandran.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chandran, A.K., Poh, L.A. & Vadakkepat, P. Real-time identification of pedestrian meeting and split events from surveillance videos using motion similarity and its applications. J Real-Time Image Proc 16, 971–987 (2019). https://doi.org/10.1007/s11554-016-0584-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-016-0584-0

Keywords

Search

Navigation