Skip to main content
SpringerLink
Log in

D2-TPred: Discontinuous Dependency for Trajectory Prediction Under Traffic Lights

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13668))

Included in the following conference series:

Abstract

A profound understanding of inter-agent relationships and motion behaviors is important to achieve high-quality planning when navigating in complex scenarios, especially at urban traffic intersections. We present a trajectory prediction approach with respect to traffic lights, D2-TPred, which uses a spatial dynamic interaction graph (SDG) and a behavior dependency graph (BDG) to handle the problem of discontinuous dependency in the spatial-temporal space. Specifically, the SDG is used to capture spatial interactions by reconstructing sub-graphs for different agents with dynamic and changeable characteristics during each frame. The BDG is used to infer motion tendency by modeling the implicit dependency of the current state on priors behaviors, especially the discontinuous motions corresponding to acceleration, deceleration, or turning direction. Moreover, we present a new dataset for vehicle trajectory prediction under traffic lights called VTP-TL. Our experimental results show that our model achieves more than 20.45% and 20.78% improvement in terms of ADE and FDE, respectively, on VTP-TL as compared to other trajectory prediction algorithms. The dataset and code are available at: https://github.com/VTP-TL/D2-TPred.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Agrim, G., Justin, J., Li, F.F., Silvio, S.: Social GAN: socially acceptable trajectories with generative adversarial networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2255–2264 (2018)

    Google Scholar 

  2. Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: human trajectory prediction in crowded spaces. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 961–971 (2016). https://doi.org/10.1109/CVPR.2016.110

  3. Angelos, M., Rohan, C., Dinesh, M.: B-GAP: behavior-guided action prediction for autonomous navigation. arXiv preprint arXiv:2011.03748 (2020)

  4. Bai, H., Cai, S., Ye, N., Hsu, D., Lee, W.S.: Intention-aware online POMDP planning for autonomous driving in a crowd. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 454–460 (2015)

    Google Scholar 

  5. Bock, J., Krajewski, R., Moers, T., Runde, S., Vater, L., Eckstein, L.: The inD dataset: a drone dataset of naturalistic road user trajectories at German intersections. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1929–1934 (2019). https://doi.org/10.1109/IV47402.2020.9304839

  6. Chandra, R., Bhattacharya, U., Bera, A., Manocha, D.: TraPHic: trajectory prediction in dense and heterogeneous traffic using weighted interactions. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8475–8484 (2019). https://doi.org/10.1109/CVPR.2019.00868

  7. Chandra, R., Bhattacharya, U., Mittal, T., Bera, A., Manocha, D.: CMetric: a driving behavior measure using centrality functions. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2035–2042 (2020)

    Google Scholar 

  8. Chandra, R., et al.: Forecasting trajectory and behavior of road-agents using spectral clustering in graph-LSTMs. IEEE Robot. Autom. Lett. 5(3), 4882–4890 (2020)

    Article  Google Scholar 

  9. Chang, M.F., et al.: Argoverse: 3D tracking and forecasting with rich maps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8748–8757 (2019)

    Google Scholar 

  10. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3213–3223 (2016)

    Google Scholar 

  11. Dendorfer, P., Oep, A., Laura, L.T.: Goal-GAN: multimodal trajectory prediction based on goal position estimation. In: Computer Vision - ACCV 2020 (2021)

    Google Scholar 

  12. Deo, N., Trivedi, M.M.: Convolutional social pooling for vehicle trajectory prediction. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1468–1476 (2018)

    Google Scholar 

  13. Dirk, H., Peter, M.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282–4286 (1995)

    Article  Google Scholar 

  14. Dzmitry, B., Kyunghyun, C., Yoshua, B.: Neural machine translation by jointly learning to align and translate. arXiv:1409.0473v7 (2014)

  15. Fang, L., Jiang, Q., Shi, J., Zhou, B.: TPNet: trajectory proposal network for motion prediction. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6796–6805 (2020). https://doi.org/10.1109/CVPR42600.2020.00683

  16. Girish, V., Anbumani, S., Anoop, N., Manmohan, C., Jawahar, C.V.: IDD: a dataset for exploring problems of autonomous navigation in unconstrained environments. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1743–1751 (2021)

    Google Scholar 

  17. Giuliari, F., Hasan, I., Cristani, M., Galasso, F.: Transformer networks for trajectory forecasting. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 10335–10342 (2020)

    Google Scholar 

  18. Hang, Z., Jiyang, G., Tian, L., Chen, S., Benjamin, S., Balakrishnan, V.: TNT: target-driven trajectory prediction. arXiv:2008.08294v2 (2020)

  19. Hasan, I., et al.: Forecasting people trajectories and head poses by jointly reasoning on tracklets and vislets. IEEE Trans. Pattern Anal. Mach. Intell. 43(4), 1267–1278 (2021)

    Article  Google Scholar 

  20. Holger, C., Varun, K.R.B., Lang, A.H., Sourabh, V.: nuScenes: a multimodal dataset for autonomous driving. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11621–11631 (2020)

    Google Scholar 

  21. Houston, J., Zuidhof, G., Bergamini, L., Ye, Y., Ondruska, P.: One thousand and one hours: self-driving motion prediction dataset. In: Conference on Robot Learning (CoRL) (2020)

    Google Scholar 

  22. Hu, Y., Chen, S., Zhang, Y., Gu, X.: Collaborative motion prediction via neural motion message passing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6318–6327 (2020)

    Google Scholar 

  23. Huang, Y., Bi, H., Li, Z., Mao, T., Wang, Z.: STGAT: modeling spatial-temporal interactions for human trajectory prediction. In: 2019 International Conference in Computer Vision, pp. 6272–6281 (2019)

    Google Scholar 

  24. Jiachen, L., Fan, Y., Tomizuka, M., Choi., C.: EvolveGraph: multi-agent trajectory prediction with dynamic relational reasoning. In: Proceedings of the Neural Information Processing Systems (NeurIPS), pp. 1–18 (2020)

    Google Scholar 

  25. Lee, N., Choi, W., Vernaza, P., Choy, C.B., Torr, P.H.S., Chandraker, M.: DESIRE: distant future prediction in dynamic scenes with interacting agents. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2165–2174 (2017). https://doi.org/10.1109/CVPR.2017.233

  26. Liang, J., Jiang, L., Hauptmann, A.: SimAug: learning robust representations from 3D simulation for pedestrian trajectory prediction in unseen cameras 2. arXiv preprint arXiv:2004.02022 (2020)

  27. Luo, Y., Cai, P., Bera, A., Hsu, D., Lee, W.S., Manocha, D.: PORCA: modeling and planning for autonomous driving among many pedestrians. IEEE Robot. Autom. Lett. 3(4), 3418–3425 (2018)

    Article  Google Scholar 

  28. Mo, X., Huang, Z., Xing, Y., Lv, C.: Multi-agent trajectory prediction with heterogeneous edge-enhanced graph attention network. IEEE Trans. Intell. Transp. Syst. 1–14 (2022). https://doi.org/10.1109/TITS.2022.3146300

  29. Mohamed, A., Qian, K., Elhoseiny, M., Claudel, C.: Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition, pp. 14424–14432 (2020)

    Google Scholar 

  30. Pang, B., Zhao, T., Xie, X., Wu, Y.N.: Trajectory prediction with latent belief energy-based model. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11814–11824, June 2021

    Google Scholar 

  31. Petar, V., Guillem, C., Arantxa, C., Adriana, R.: Graph attention networks. In: International Conference on Learning Representations (ICLR) (2018)

    Google Scholar 

  32. Pongsathorn, R., Takahiro, H., Masao, N.: Motion planning and control of autonomous driving intelligence system based on risk potential optimization framework. Int. J. Autom. Eng. 7(1), 53–60 (2016)

    Google Scholar 

  33. Ramin, M., Alexis, O., Mubarak, S.: Abnormal crowd behavior detection using social force model. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942 (2009)

    Google Scholar 

  34. Robicquet, A., Sadeghian, A., Alahi, A., Savarese, S.: Learning social etiquette: human trajectory understanding in crowded scenes. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 549–565. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_33

    Chapter  Google Scholar 

  35. Rohan, C., et al.: METEOR: a massive dense & heterogeneous behavior dataset for autonomous driving. arXiv preprint arXiv:2109.07648 (2021)

  36. Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Rezatofighi, H., Savarese, S.: SoPhie: an attentive GAN for predicting paths compliant to social and physical constraints. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1349–1358 (2020)

    Google Scholar 

  37. Scott, E., et al.: Large scale interactive motion forecasting for autonomous driving: the waymo open motion dataset. arXiv preprint arXiv:2104.10133 (2021)

  38. Shi, L., et al.: SGCN: sparse graph convolution network for pedestrian trajectory prediction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  39. Song, X., et al.: Pedestrian trajectory prediction based on deep convolutional LSTM network. IEEE Trans. Intell. Transp. Syst. 22(6), 3285–3302 (2021)

    Article  Google Scholar 

  40. Song, Y., Bisagno, N., Hassan, S.Z., Conci, N.: AG-GAN: an attentive group-aware GAN for pedestrian trajectory prediction. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 8703–8710 (2021)

    Google Scholar 

  41. Tim, S., Boris, I., Punarjay, C., Marco, P.: Trajectron++: multi-agent generative trajectory forecasting with heterogeneous data for control. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 683–700 (2020)

    Google Scholar 

  42. Vemula, A., Muelling, K., Oh, J.: Social attention: modeling attention in human crowds, pp. 1–7 (2018)

    Google Scholar 

  43. Vineet, K., Amir, S., Roberto, M.M., Lan, R.: Social-BiGAT: multimodal trajectory forecasting using bicycle-GAN and graph attention networks. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 137–146 (2019)

    Google Scholar 

  44. University of Waterloo: uwaterloo (2021). http://wiselab.uwaterloo.ca/waterloo-multi-agent-traffic-dataset/

  45. Wei, Z., et al.: Interaction dataset: an international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps. arXiv preprint arXiv:1910.03088 (2019)

  46. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2019)

    Article  MathSciNet  Google Scholar 

  47. Xin, L., Xiaowen, Y., Chuah, M.C.: GRIP++: enhanced graph-based interaction-aware trajectory prediction for autonomous driving. arXiv preprint arXiv:1907.07792 (2020)

  48. Xu, Y., Ren, D., Li, M., Chen, Y., Fan, M., Xia, H.: Tra2Tra: trajectory-to-trajectory prediction with a global social spatial-temporal attentive neural network. IEEE Robot. Autom. Lett. 6(2), 1574–1581 (2021)

    Article  Google Scholar 

  49. Yuexin, M., Dinesh, M., Wenping, W.: AutoRVO: local navigation with dynamic constraints in dense heterogeneous traffic. arXiv preprint arXiv:1804.02915 (2018)

  50. Yuexin, M., Xinge, Z., Sibo, Z., Ruigang, Y., Wenping, W., Dinesh, M.: TrafficPredict: trajectory prediction for heterogeneous traffic-agents. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 6120–6127 (2019)

    Google Scholar 

  51. Yuning, C., Benjamin, S., Mayank, B., Dragomir, A.: MultiPath: multiple probabilistic anchor trajectory hypotheses for behavior prediction. arXiv:1910.05449 (2019)

  52. Zernetsch, S., Kohnen, S., Goldhammer, M., Doll, K., Sick, B.: Trajectory prediction of cyclists using a physical model and an artificial neural network. In: 2016 IEEE Intelligent Vehicles Symposium (IV), pp. 833–838 (2016)

    Google Scholar 

  53. Zhang, K., Zhao, L., Dong, C., Wu, L., Zheng, L.: AI-TP: attention-based interaction-aware trajectory prediction for autonomous driving. IEEE Trans. Intell. Veh. 1 (2022). https://doi.org/10.1109/TIV.2022.3155236

  54. Zhao, L., et al.: T-GCN: a temporal graph convolutional network for traffic prediction. IEEE Trans. Intell. Transp. Syst. 21(9), 3848–3858 (2019)

    Article  Google Scholar 

  55. Zhao, Z., Liu, C.: STUGCN: a social spatio-temporal unifying graph convolutional network for trajectory prediction. In: 2021 6th International Conference on Automation, Control and Robotics Engineering (CACRE), pp. 546–550 (2021)

    Google Scholar 

  56. Zheng, F., et al.: Unlimited neighborhood interaction for heterogeneous trajectory prediction. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13148–13157 (2021). https://doi.org/10.1109/ICCV48922.2021.01292

  57. Zhengping, C., et al.: D\(^2\)-City: a large-scale dashcam video dataset of diverse traffic scenarios. arXiv preprint arXiv:1904.01975 (2019)

  58. Zhou, J., et al.: Graph neural networks: a review of methods and applications. AI Open 1, 57–81 (2020). https://doi.org/10.1016/j.aiopen.2021.01.001

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China with Grant No. 61772474 and 62036010, Zhengzhou Major Science and Technology Project with Grant No. 2021KJZX0060-6. We thank all the reviewers for their valuable suggestions.

Author information

Authors and Affiliations

  1. School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, China

    Yuzhen Zhang, Weizhi Guo, Pei Lv & Mingliang Xu

  2. Henan Institute of Advanced Technology, Zhengzhou University, Zhengzhou, China

    Wentong Wang

  3. State Key Lab of CAD &CG, Zhejiang University, Hangzhou, Zhejiang, China

    Wei Chen

  4. Department of Computer Science, University of Maryland, College Park, MD, USA

    Dinesh Manocha

Authors
  1. Yuzhen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wentong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weizhi Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pei Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mingliang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dinesh Manocha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pei Lv .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, Y. et al. (2022). D2-TPred: Discontinuous Dependency for Trajectory Prediction Under Traffic Lights. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13668. Springer, Cham. https://doi.org/10.1007/978-3-031-20074-8_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20074-8_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20073-1

  • Online ISBN: 978-3-031-20074-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation