Skip to main content
SpringerLink
Log in

Fast Segmentation-Based Object Tracking Model for Autonomous Vehicles

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12453))

Abstract

On-road object tracking is a critical module for both Advanced Driving Assistant System (ADAS) and autonomous vehicles. Commonly, this function can be achieved through single vehicle sensors, such as a camera or LiDAR. Consider the low cost and wide application of optical cameras, a simple image segmentation-based on-road object tracking model is proposed. Different from the detection-based tracking with bounding box, our model improves tracking performance from the following three aspects: 1) the Positional Normalization (PONO) feature is used to enhance the target outline with common convolutional layers. 2) The inter-frame correlation of each target used for tracking relies on mask, this helps the model reducing the influences caused by the background around the targets. 3) By using a bidirectional LSTM module capable of capturing timing correlation information, the forward and reverse matching of the targets in consecutive frames is performed. We also evaluate the presented model on the KITTI MOTS (Multi-Object and Segmentation) task which collected from out door environment for autonomous vehicle. Results show that our model is three times faster than Track RCNN with slightly drop on sMOTSA, and is more suitable for deployment on vehicular low-power edge computing equipment.

Supported by Hangzhou Innovation Institution, Beihang University.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    We provide code online at https://github.com/XYunaaa/Fast-Segmentation-based-Object-Tracking-Model.

  2. 2.

    https://www.vision.rwth-aachen.de/page/mots.

References

  1. Porzi, L., Hofinger, M., Ruiz, I., Serrat, J., Bulò, S.R., Kontschieder, P.: Learning multi-object tracking and segmentation from automatic annotations. arXiv preprint arXiv:1912.02096 (2019)

  2. Osep, A., Voigtlaender, P., Weber, M., Luiten, J., Leibe, B.: 4D generic video object proposals. arXiv preprint arXiv:1901.09260 (2019)

  3. Voigtlaender, P., et al.: Mots: multi-object tracking and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7942–7951 (2019)

    Google Scholar 

  4. Li, X., Weiming, H., Shen, C., Zhang, Z., Dick, A., Van Den Hengel, A.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. (TIST) 4(4), 1–48 (2013)

    Article  Google Scholar 

  5. Ward, J.S., Barker, A.: Undefined by data: a survey of big data definitions. arXiv preprint arXiv:1309.5821 (2013)

  6. Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv preprint arXiv:1504.01942 (2015)

  7. Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: Mot16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831, 2016

  8. Chen, Y., Jing, L., Vahdani, E., Zhang, L., He, M., Tian, Y.: Multi-camera vehicle tracking and re-identification on AI city challenge 2019. In: Proceedings of CVPR Workshops (2019)

    Google Scholar 

  9. Milan, A., Schindler, K., Roth, S.: Challenges of ground truth evaluation of multi-target tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 735–742 (2013)

    Google Scholar 

  10. Shi, W., Alawieh, M.B., Li, X., Yu, H.: Algorithm and hardware implementation for visual perception system in autonomous vehicle: a survey. Integr. 59, 148–156 (2017)

    Article  Google Scholar 

  11. Leal-Taixé, L., Milan, A., Schindler, K., Cremers, D., Reid, I., Roth, S.: Tracking the trackers: an analysis of the state of the art in multiple object tracking. arXiv preprint arXiv:1704.02781 (2017)

  12. Ouyang, Z., Niu, J., Liu, Y., Guizani, M.: Deep CNN-based real-time traffic light detector for self-driving vehicles. IEEE Trans. Mob. Comput. 19(2), 300–313 (2019)

    Article  Google Scholar 

  13. Luo, W., et al.: Multiple object tracking: a literature review. arXiv preprint arXiv:1409.7618 (2014)

  14. Ciaparrone, G., Sánchez, F.L., Tabik, S., Troiano, L., Tagliaferri, R., Herrera, F.: Deep learning in video multi-object tracking: a survey. Neurocomputing 381, 61–88 (2020)

    Article  Google Scholar 

  15. Li, B., Wu, F., Weinberger, K.Q., Belongie, S.: Positional normalization. In: Advances in Neural Information Processing Systems, pp. 1620–1632 (2019)

    Google Scholar 

  16. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

    Google Scholar 

  17. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  18. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-ResNet and the impact of residual connections on learning. In: 31st AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  19. Wang, L., Xu, L., Kim, M.Y., Rigazico, L., Yang, M.H.: Online multiple object tracking via flow and convolutional features. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3630–3634. IEEE (2017)

    Google Scholar 

  20. Wang, Q., Zhang, L., Bertinetto, L., Hu, W., Torr, P.H.: Fast online object tracking and segmentation: a unifying approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1328–1338 (2019)

    Google Scholar 

  21. Zhao, D., Hao, F., Xiao, L., Tao, W., Dai, B.: Multi-object tracking with correlation filter for autonomous vehicle. Sensors 18(7), 2004 (2018)

    Article  Google Scholar 

  22. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE (2016)

    Google Scholar 

  23. Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J.: POI: multiple object tracking with high performance detection and appearance feature. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 36–42. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_3

    Chapter  Google Scholar 

  24. Chen, J., Sheng, H., Zhang, Y., Xiong, Z.: Enhancing detection model for multiple hypothesis tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 18–27 (2017)

    Google Scholar 

  25. Tan, X., et al.: Multi-camera vehicle tracking and re-identification based on visual and spatial-temporal features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 275–284 (2019)

    Google Scholar 

  26. Zhang, S., et al.: Tracking persons-of-interest via adaptive discriminative features. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 415–433. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_26

    Chapter  Google Scholar 

  27. Kieritz, H., Hubner, W., Arens, M.: Joint detection and online multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1459–1467 (2018)

    Google Scholar 

  28. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1385–1392 (2013)

    Google Scholar 

  29. Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: Atom: accurate tracking by overlap maximization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4660–4669 (2019)

    Google Scholar 

  30. Maksai, A., Fua, P.: Eliminating exposure bias and metric mismatch in multiple object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4639–4648 (2019)

    Google Scholar 

  31. Li, X., Ma, C., Wu, B., He, Z., Yang, M.H.: Target-aware deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1369–1378 (2019)

    Google Scholar 

  32. Fan, H., Ling, H.: Siamese cascaded region proposal networks for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7952–7961 (2019)

    Google Scholar 

  33. Gao, J., Zhang, T., Xu, C.: Graph convolutional tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4649–4659 (2019)

    Google Scholar 

  34. Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., Yan, J.: Siamrpn++: evolution of siamese visual tracking with very deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4282–4291 (2019)

    Google Scholar 

  35. Wang, G., Luo, C., Xiong, Z., Zeng, W.: SPM-tracker: series-parallel matching for real-time visual object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3643–3652 (2019)

    Google Scholar 

  36. Milan, A., Rezatofighi, S.H., Dick, A., Reid, I., Schindler, K.: Online multi-target tracking using recurrent neural networks. In: 31st AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  37. Kim, C., Li, F., Rehg, J.M.: Multi-object tracking with neural gating using bilinear LSTM. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 200–215 (2018)

    Google Scholar 

  38. Wang, N., Song, Y., Ma, C., Zhou, W., Liu, W., Li, H.: Unsupervised deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1308–1317 (2019)

    Google Scholar 

  39. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

Download references

Acknowledgment

This work has been supported by National Natural Science Foundation of China (61772060, 61976012), Qianjiang Postdoctoral Foundation (2020-Y4-A-001), and CERNET Innovation Project (NGII20170315).

Author information

Authors and Affiliations

  1. Hangzhou Innovation Institution, Beihang University, Chuanghui Street #18, Binjiang, Hangzhou, 310000, Zhejiang, China

    Xiaoyun Dong, Jianwei Niu, Jiahe Cui, Zongkai Fu & Zhenchao Ouyang

  2. State Key Laboratory of Virtual Reality Technology and Systems, Beijing, China

    Xiaoyun Dong, Jianwei Niu, Jiahe Cui, Zongkai Fu & Zhenchao Ouyang

  3. Beijing Advanced Innovation Center for Big Data and Brain Computing (BDBC) Beihang University, Xueyuan Road #37, Haidian, Beijing, 100191, China

    Xiaoyun Dong, Jianwei Niu, Jiahe Cui & Zongkai Fu

  4. Nanhu Laboratory, Jiaxin, 314000, Zhejiang, China

    Zhenchao Ouyang

  5. Zhengzhou University Research Institute of Industrial Technology, Zhengzhou University, Zhengzhou, 450001, China

    Jianwei Niu

Authors
  1. Xiaoyun Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianwei Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiahe Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zongkai Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhenchao Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenchao Ouyang .

Editor information

Editors and Affiliations

  1. Columbia University, New York, NY, USA

    Meikang Qiu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dong, X., Niu, J., Cui, J., Fu, Z., Ouyang, Z. (2020). Fast Segmentation-Based Object Tracking Model for Autonomous Vehicles. In: Qiu, M. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2020. Lecture Notes in Computer Science(), vol 12453. Springer, Cham. https://doi.org/10.1007/978-3-030-60239-0_18

Download citation

Publish with us

Policies and ethics

Search

Navigation