Skip to main content
SpringerLink
Log in

A Hierarchical LSTM-Based Vehicle Trajectory Prediction Method Considering Interaction Information

  • Published:
Automotive Innovation Aims and scope Submit manuscript

Abstract

Trajectory prediction is an essential component in autonomous driving systems, as it can forecast the future movements of surrounding vehicles, thereby enhancing the decision-making and planning capabilities of autonomous driving systems. Traditional models relying on constant acceleration and constant velocity often experience a reduction in prediction accuracy as the forecasted timeframe extends. This limitation makes it challenging to meet the demands for medium to long-term trajectory prediction. Conversely, data-driven models, particularly those based on Long Short-Term Memory (LSTM) neural networks, have demonstrated superior performance in medium to long-term trajectory prediction. Therefore, this study introduces a hierarchical LSTM-based method for vehicle trajectory prediction. Considering the difficulty of using a single LSTM model to predict trajectories for all driving intentions, the trajectory prediction task is decomposed into three sequential steps: driving intention prediction, lane change time prediction, and trajectory prediction. Furthermore, given that the driving intent and trajectory of a vehicle are always subject to the influence of the surrounding traffic flow, the predictive model proposed in this paper incorporates the interactional information of neighboring vehicle movements into the model input. The proposed method is trained and validated on the real vehicle trajectory dataset Next Generation Simulation. The results show that the proposed hierarchical LSTM method has a lower prediction error compared to the integral LSTM model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Abbreviations

LSTM:

Long Short-Term Memory

NGSIM:

Next Generation Simulation

RNN:

Recurrent neural network

References

  1. Post, J., Veldstra, J., Ünal, A.: Acceptability and acceptance of connected automated vehicles: a literature review and focus groups. Paper presented at the Proceedings of the 5th International Conference on Computer-Human Interaction Research and Applications, Valletta, (2021). https://doi.org/10.5220/0010719200003060

  2. Liu, J., Mao, X., Fang, Y., Zhu, D., Meng, M. Q. -H.: A Survey on deep-learning approaches for vehicle trajectory prediction in autonomous driving. Paper presented at 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO), Sanya, 27–31 December 2021. https://doi.org/10.1109/ROBIO54168.2021.9739407

  3. Leon, F., Gavrilescu, M.: A review of tracking and trajectory prediction methods for autonomous driving. Mathematics 9(6), 660 (2021). https://doi.org/10.3390/math9060660

    Article  Google Scholar 

  4. Li, S.E., Peng, H., Li, K., Wang, J.: Minimum fuel control strategy in automated car-following scenarios. IEEE Trans. Veh. Technol.Veh. Technol. 61(3), 998–1007 (2012)

    Article  Google Scholar 

  5. Kamal, M., Taguchi, S., Yoshimura, T.: Efficient driving on multilane roads under a connected vehicle environment. IEEE Trans. Intell. Transp. Syst.Intell. Transp. Syst. 17(9), 2541–2551 (2016)

    Article  Google Scholar 

  6. Lytrivis P., Thomaidis G., Amditis, A.: Cooperative path prediction in vehicular environments. Paper presented at 11th International IEEE Conference on Intelligent Transportation Systems, Beijing, 12–15 October 2008. https://doi.org/10.1109/ITSC.2008.4732629

  7. Barth A., Franke U.: Where will the oncoming vehicle be the next second. Paper presented at IEEE Intelligent Vehicles Symposium, Eindhoven, Netherlands, 04–06 June 2008

  8. Guo, C., Sentouh, C., Soualmi B., Haué, J. -B., Popieul, J. -C.: Adaptive vehicle longitudinal trajectory prediction for automated highway driving. Paper presented at IEEE Intelligent Vehicles Sym-posium (IV), Gothenburg, 19–22 June 2016. https://doi.org/10.1109/IVS.2016.7535555

  9. Houenou, A., Bonnifait, P., Cherfaoui, V., Yao, W.: Vehicle trajectory prediction based on motion model and maneuver recognition. Paper presented at 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, 03–07 November 2013 https://doi.org/10.1109/IROS.2013.6696982

  10. Jin, B., Jiu, B., Su, T., et al.: Switched Kalman filter-interacting multiple model algorithm based on optimal autoregressive model for manoeuvring target tracking. IET Radar Sonar Navig.Navig. 9(2), 199–209 (2015). https://doi.org/10.1049/iet-rsn.2014.0142

    Article  Google Scholar 

  11. Wang, Y., Liu, Z., Zuo, Z., et al.: Trajectory planning and safety assessment of autonomous vehicles based on motion prediction and model predictive control. IEEE Trans. Veh. Technol.Veh. Technol. 68(9), 8546–8556 (2019). https://doi.org/10.1109/TVT.2019.2930684

    Article  Google Scholar 

  12. Benterki A., Judalet V., Choubeila M., Boukhnifer M.: Long-term prediction of vehicle trajectory using recurrent neural networks. Paper presented at 45th Annual Conference of the IEEE Industrial Electronics Society, Lisbon, 14–17 October 2019. https://doi.org/10.1109/IECON.2019.8927604

  13. Zhang, H., Wang, Z., Liu, D.: A comprehensive review of stability analysis of continuous-time recurrent neural networks. IEEE Transact. Neural Netw. Learn. Syst. 25(7), 1229–1262 (2014). https://doi.org/10.1109/TNNLS.2014.2317880

    Article  ADS  Google Scholar 

  14. Sepp, H., Jürgen, S.: Long short-term memory. Neural Comput.Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  15. Gers, F.: Long short-term memory in recurrent neural networks. École Polytechnique Fédérale de Lausanne, Switzerland, (2001)

  16. Yu, Y., Si, X., Hu, C., Zhang, J.: A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput.Comput. 31(7), 1235–1270 (2019). https://doi.org/10.1162/neco_a_01199

    Article  MathSciNet  Google Scholar 

  17. Lihua, J., Huiqun, X., Guobin, L.: LSTM-based attentional embedding for english machine translation. Sci. Program. 2022, 1–8 (2022). https://doi.org/10.1155/2022/3909726

    Article  Google Scholar 

  18. Chen, K., Song, X., Yu, H.: Conv-LSTM: pedestrian trajectory prediction in crowded scenarios. Paper presented at Communications in Computer and Information Science, Singapore, 2019. https://doi.org/10.1007/978-981-15-1078-6_3

  19. Xue, H., Huynh, D. Q., Reynolds, M.: SS-LSTM: A hierarchical LSTM model for pedestrian trajectory prediction. Paper presented at 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, 12–15 March 2018. https://doi.org/10.1109/WACV.2018.00135

  20. Xiao, Y., Nian, Q.: Vehicle location prediction based on spatiotemporal feature transformation and hybrid LSTM neural network. Information 11(2), 84 (2020). https://doi.org/10.3390/info11020084

    Article  Google Scholar 

  21. Wang, S., Zhao, P., Yu, B., Huang, W., Liang, H.: Vehicle trajectory prediction by knowledge-driven LSTM network in urban environments. J. Adv. Transp. 2020, 8894060 (2020). https://doi.org/10.1155/2020/8894060

    Article  Google Scholar 

  22. Horng, G.-J., Huang, Y.-C., Yin, Z.-X.: Using bidirectional long-term memory neural network for trajectory prediction of large inner wheel routes. Sustainability 14(10), 5935 (2022). https://doi.org/10.3390/su14105935

    Article  Google Scholar 

  23. Deo, N., Trivedi, M. M.: Convolutional social pooling for vehicle trajectory prediction. Paper presented at 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, 18–22 June 2018. https://doi.org/10.1109/CVPRW.2018.00196

  24. Dai, S., Li, Z., Li, L., Zheng, N., Wang, S.: A flexible and explainable vehicle motion prediction and inference framework combining semi-supervised AOG and ST-LSTM. IEEE Trans. Intell. Transp. Syst.Intell. Transp. Syst. 23(2), 840–860 (2022). https://doi.org/10.1109/TITS.2020.3016304

    Article  Google Scholar 

  25. Zhang, K., Li, L.: Explainable multimodal trajectory prediction using attention models. Transp. Res. Part C: Emerg Technol. 143, 103829 (2022). https://doi.org/10.1016/j.trc.2022.103829

    Article  Google Scholar 

  26. Lin, L., Li, W., Bi, H., Qin, L.: Vehicle trajectory prediction using LSTMs with Spatial-Temporal attention mechanisms. IEEE Intell. Transp. Syst. Mag.Intell. Transp. Syst. Mag. 14(2), 197–208 (2022). https://doi.org/10.1109/MITS.2021.3049404

    Article  Google Scholar 

  27. Benrachou, D.E., Glaser, S., Elhenawy, M., Rakotonirainy, A.: Use of social interaction and intention to improve motion prediction within automated vehicle framework: a review. IEEE Trans. Intell. Transp. Syst.Intell. Transp. Syst. 23(12), 22807–22837 (2022). https://doi.org/10.1109/TITS.2022.3207347

    Article  Google Scholar 

  28. Dong, C., Chen, Y., Dolan, J. M.: Interactive trajectory prediction for autonomous driving via recurrent meta induction neural network. Paper presented at 2019 International Conference on Robotics and Automation (ICRA), Montreal, 20–24 May (2019). https://doi.org/10.1109/ICRA.2019.8794392.

  29. Dai, S., Li, L., Li, Z.: Modeling vehicle interactions via modified LSTM models for trajectory prediction. IEEE Access 7, 38287–38296 (2019). https://doi.org/10.1109/ACCESS.2019.2907000

    Article  Google Scholar 

  30. Xie, G., Gao, H., Qian, L., Huang, B., Li, K., Wang, J.: Vehicle trajectory prediction by integrating physics- and maneuver-based approaches using interactive multiple models. IEEE Trans. Industr. Electron.Industr. Electron. 65(7), 5999–6008 (2018). https://doi.org/10.1109/TIE.2017.2782236

    Article  Google Scholar 

  31. Nilsson, J., Brännström, M., Coelingh, E., Fredriksson, J.: Longitudinal and lateral control for automated lane change maneuvers. Paper presented at 2015 American Control Conference (ACC), Chicago, 01–03 (2015). https://doi.org/10.1109/ACC.2015.7170929

  32. Benterki, A., Boukhnifer, M., Judalet, V., Maaoui, C.: Artificial intelligence for vehicle behavior anticipation: hybrid approach based on maneuver classification and trajectory prediction. IEEE Access 8, 56992–57002 (2020). https://doi.org/10.1109/ACCESS.2020.2982170

    Article  Google Scholar 

  33. Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Transact Neural Netw Learn Syst. 28(10), 2222–2232 (2017). https://doi.org/10.1109/TNNLS.2016.2582924

    Article  MathSciNet  Google Scholar 

  34. Benjamin, C., Lizhe, L.: A critical evaluation of the next generation simulation (NGSIM) vehicle trajectory dataset. Transp. Res. Part B: Methodol. 105, 362–377 (2017). https://doi.org/10.1016/j.trb.2017.09.018

    Article  Google Scholar 

  35. Wu, X., Gao, X., Zhang, W., Luo, R., Wang, J.: Learning over categorical data using counting features: with an application on click-through rate estimation. Paper presented at Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data (DLP-KDD '19), Association for Computing Machinery, New York, (2019). https://doi.org/10.1145/3326937.3341260

  36. Kline, D.M., Berardi, V.L.: Revisiting squared-error and cross-entropy functions for training neural network classifiers. Neural Comput. Appl.Comput. Appl. 14, 310–318 (2005). https://doi.org/10.1007/s00521-005-0467-y

    Article  Google Scholar 

  37. Diederik P. K., Jimmy B.: Adam: a method for stochastic optimization. Paper presented at the 3rd International Conference for Learning Representations, San Diego, (2015). https://doi.org/10.48550/arXiv.1412.6980

Download references

Acknowledgements

This work was supported by the Jilin Province Science and Technology Development Program (20210301023GX).

Author information

Authors and Affiliations

  1. State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun, 130012, China

    Haitao Min, Xiaoyong Xiong, Pengyu Wang & Zhaopu Zhang

Authors
  1. Haitao Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyong Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pengyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhaopu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhaopu Zhang.

Ethics declarations

Conflict of interest

No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Min, H., Xiong, X., Wang, P. et al. A Hierarchical LSTM-Based Vehicle Trajectory Prediction Method Considering Interaction Information. Automot. Innov. 7, 71–81 (2024). https://doi.org/10.1007/s42154-023-00261-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42154-023-00261-0

Keywords

Search

Navigation