Skip to main content
SpringerLink
Log in

Key-Track: A Lightweight Scalable LSTM-based Pedestrian Tracker for Surveillance Systems

  • Conference paper
  • First Online:
Image Analysis and Recognition (ICIAR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11663))

Included in the following conference series:

Abstract

There has been a growing interest in leveraging state of the art deep learning techniques for tracking objects in recent years. Most of this work focuses on using redundant appearance models for predicting object tracklets for the next frame. Moreover, not much work has been done to explore the sequence learning properties of Long Short Term Memory (LSTM) Neural Networks for object tracking in video sequences. In this work we propose a novel LSTM tracker, Key-Track, which effectively learns the spatial and temporal behavior of pedestrians after analyzing movement patterns of human key-points provided to it by OpenPose [3]. We train Key-Track on single person sequences that we curated from the Duke Multi-target Multi-Camera (Duke-MTMC) [26] dataset and scale it to track multiple people at run-time, further testing its scalability. We report our results on the Duke-MTMC dataset for different time-series sequence lengths we feed to Key-Track and find three as the optimum time-step sequence length producing the highest Average Overlap Score (AOS). We further present our qualitative analysis on these different time-series sequence lengths producing different results depending on the type of video sequence. The total observed size of Key-Track is under 1 megabytes which paves its way into mobile devices for the purpose of tracking in real-time.

This research was supported by the National Science Foundation (NSF) under Award No. 1831795.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Project page: https://github.com/TeCSAR-UNCC/key-track.

References

  1. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional siamese networks for object tracking. CoRR abs/1606.09549 (2016). http://arxiv.org/abs/1606.09549

  2. Bontemps, L., Cao, V.L., McDermott, J., Le-Khac, N.-A.: Collective anomaly detection based on long short-term memory recurrent neural networks. In: Dang, T.K., Wagner, R., Küng, J., Thoai, N., Takizawa, M., Neuhold, E. (eds.) FDSE 2016. LNCS, vol. 10018, pp. 141–152. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48057-2_9

    Chapter  Google Scholar 

  3. Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. arXiv preprint. arXiv:1812.08008 (2018)

  4. Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., Yu, N.: Online multi-object tracking using cnn-based single object tracker with spatial-temporal attention mechanism. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 October 2017, pp. 4846–4855 (2017). https://doi.org/10.1109/ICCV.2017.518

  5. Cui, Z., Ke, R., Wang, Y.: Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction. CoRR abs/1801.02143 (2018). http://arxiv.org/abs/1801.02143

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR09 (2009)

    Google Scholar 

  7. Dequaire, J., Ondruska, P., Rao, D., Wang, D.Z., Posner, I.: Deep tracking in the wild: end-to-end tracking using recurrent neural networks. Int. J. Robot. Res. 37(4–5), 492–512 (2018). https://doi.org/10.1177/0278364917710543

    Article  Google Scholar 

  8. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vision 88(2), 303–338 (2010)

    Article  Google Scholar 

  9. Fang, K.: Track-RNN: joint detection and tracking using recurrent neural networks

    Google Scholar 

  10. Feichtenhofer, C., Pinz, A., Zisserman, A.: Detect to track and track to detect. In: IEEE International Conference on Computer Vision (2017)

    Google Scholar 

  11. Girdhar, R., Gkioxari, G., Torresani, L., Paluri, M., Tran, D.: Detect-and-track: efficient pose estimation in videos. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 350–359, June 2018. https://doi.org/10.1109/CVPR.2018.00044

  12. Gordon, D., Farhadi, A., Fox, D.: Re3 : Real-time recurrent regression networks for object tracking. CoRR abs/1705.06368 (2017). http://arxiv.org/abs/1705.06368

  13. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  14. Kokul, T., Fookes, C., Sridharan, S., Ramanan, A., Pinidiyaarachchi, U.A.J.: Gate connected convolutional neural network for object tracking. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 2602–2606, September 2017. https://doi.org/10.1109/ICIP.2017.8296753

  15. Lin, P., Mo, X., Lin, G., Ling, L., Wei, T., Luo, W.: A news-driven recurrent neural network for market volatility prediction. In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pp. 776–781, November 2017. https://doi.org/10.1109/ACPR.2017.35

  16. Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2873–2879 (2016). http://www.ijcai.org/Abstract/16/408

  17. Luo, W., et al.: Multiple object tracking: a literature review. arXiv preprint. arXiv:1409.7618 (2014)

  18. Martinez, J., Black, M.J., Romero, J.: On human motion prediction using recurrent neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4674–4683, July 2017. https://doi.org/10.1109/CVPR.2017.497

  19. Masala, G.L., Golosio, B., Tistarelli, M., Grosso, E.: 2D recurrent neural networks for robust visual tracking of non-rigid bodies. In: Proceedings of Engineering Applications of Neural Networks - 17th International Conference, EANN 2016, Aberdeen, UK, 2–5 September 2016, pp. 18–34 (2016). https://doi.org/10.1007/978-3-319-44188-7_2

    Chapter  Google Scholar 

  20. Milan, A., Leal-Taixé, L., Reid, I.D., Roth, S., Schindler, K.: MOT16: a benchmark for multi-object tracking. CoRR abs/1603.00831 (2016). http://arxiv.org/abs/1603.00831

  21. Milan, A., Rezatofighi, S.H., Dick, A., Reid, I., Schindler, K.: Online multi-target tracking using recurrent neural networks. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  22. Ning, G., Zhang, Z., Huang, C., He, Z., Ren, X., Wang, H.: Spatially supervised recurrent convolutional neural networks for visual object tracking. CoRR abs/1607.05781 (2016). http://arxiv.org/abs/1607.05781

  23. Oh, D.H., Shah, Z., Jang, G.: Line-break prediction of hanmun text using recurrent neural networks. In: 2017 International Conference on Information and Communication Technology Convergence (ICTC), pp. 720–724, Oct 2017. https://doi.org/10.1109/ICTC.2017.8190763

  24. Ray, A., Rajeswar, S., Chaudhury, S.: Text recognition using deep BLSTM networks. In: 2015 Eighth International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–6, January 2015. https://doi.org/10.1109/ICAPR.2015.7050699

  25. Reid, D.: An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 24(6), 843–854 (1979)

    Article  Google Scholar 

  26. Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision Workshop on Benchmarking Multi-Target Tracking (2016)

    Google Scholar 

  27. Ristani, E., Tomasi, C.: Features for multi-target multi-camera tracking and re-identification. In: Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  28. Sadeghian, A., Alahi, A., Savarese, S.: Tracking the untrackable: learning to track multiple cues with long-term dependencies. In: The IEEE International Conference on Computer Vision (ICCV), October 2017

    Google Scholar 

  29. Schulter, S., Vernaza, P., Choi, W., Chandraker, M.: Deep network flow for multi-object tracking. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2730–2739, July 2017. https://doi.org/10.1109/CVPR.2017.292

  30. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  31. Wang, Q., Huang, H.: Learning of recurrent convolutional neural networks with applications in pattern recognition. In: 2017 36th Chinese Control Conference (CCC), pp. 4135–4139, July 2017. https://doi.org/10.23919/ChiCC.2017.8028007

Download references

Author information

Authors and Affiliations

  1. University of North Carolina at Charlotte, Charlotte, USA

    Pratik Kulkarni, Shrey Mohan, Samuel Rogers & Hamed Tabkhi

Authors
  1. Pratik Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shrey Mohan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Samuel Rogers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hamed Tabkhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pratik Kulkarni .

Editor information

Editors and Affiliations

  1. University of Waterloo, Waterloo, ON, Canada

    Fakhri Karray

  2. University of Porto, Porto, Portugal

    Aurélio Campilho

  3. University of Waterloo, Waterloo, ON, Canada

    Alfred Yu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kulkarni, P., Mohan, S., Rogers, S., Tabkhi, H. (2019). Key-Track: A Lightweight Scalable LSTM-based Pedestrian Tracker for Surveillance Systems. In: Karray, F., Campilho, A., Yu, A. (eds) Image Analysis and Recognition. ICIAR 2019. Lecture Notes in Computer Science(), vol 11663. Springer, Cham. https://doi.org/10.1007/978-3-030-27272-2_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27272-2_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27271-5

  • Online ISBN: 978-3-030-27272-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation