Applied Intelligence

, Volume 48, Issue 10, pp 3523–3537 | Cite as

Multivariate time series prediction of lane changing behavior using deep neural network

  • Jun Gao
  • Yi Lu Murphey
  • Honghui Zhu


Many real world pattern classification problems involve the process and analysis of multiple variables in temporal domain. This type of problem is referred to as Multivariate Time Series (MTS) problem. It remains a challenging problem due to the nature of time series data: high dimensionality, large data size and updating continuously. In this paper, we use three types of physiological signals from the driver to predict lane changes before the event actually occurs. These are the electrocardiogram (ECG), galvanic skin response (GSR), and respiration rate (RR) and were determined, in prior studies, to best reflect a driver’s response to the driving environment. A novel Group-wise Convolutional Neural Network, MTS-GCNN model is proposed for MTS pattern classification. In our MTS-GCNN model, we present a new structure learning algorithm in training stage. The algorithm exploits the covariance structure over multiple time series to partition input volume into groups, then learns the MTS-GCNN structure explicitly by clustering input sequences with spectral clustering. Different from other feature-based classification approaches, our MTS-GCNN can select and extract the suitable internal structure to generate temporal and spatial features automatically by using convolution and down-sample operations. The experimental results showed that, in comparison to other state-of-the-art models, our MTS-GCNN performs significantly better in terms of prediction accuracy.


Multivariate time series Lane change prediction MTS-GCNN Spectral clustering 



This research is supported in part by a University Research Grant from Ford Motor Company.


  1. 1.
    Kumar P, Perrollaz M, Lefevre S et al. (2013) Learning-based approach for online lane change intention prediction. In: Intelligent vehicles symposium. IEEE, pp 797–802Google Scholar
  2. 2.
    Morris B, Doshi A, Trivedi M (2011) Lane change intent prediction for driver assistance: on-road design and evaluation. In: Intelligent vehicles symposium. IEEE, pp 895–901Google Scholar
  3. 3.
    Bevly D et al. (2016) Lane change and merge maneuvers for connected and automated vehicles: a survey. IEEE Trans Intell Veh 1(1):105–120CrossRefGoogle Scholar
  4. 4.
    Kim I H, Bong J H, Park J et al. (2017) Prediction of drivers intention of lane change by augmenting sensor information using machine learning techniques. Sensors 17(6):1350CrossRefGoogle Scholar
  5. 5.
    Butakov V A, Ioannou P (2015) Personalized driver/vehicle lane change models for ADAS. IEEE Trans Veh Technol 64(10):4422–4431CrossRefGoogle Scholar
  6. 6.
    Krizhevsky A, Sutskever I, Hinton G E (2012) ImageNet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105Google Scholar
  7. 7.
    Ji S, Xu W, Yang M et al. (2013) 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231CrossRefGoogle Scholar
  8. 8.
    He K, Zhang X, Ren S et al. (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916CrossRefGoogle Scholar
  9. 9.
    Ren S, He K, Girshick R et al. (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149CrossRefGoogle Scholar
  10. 10.
    Tran D, Sheng W, Liu L et al. (2015) A hidden Markov model based driver intention prediction system. In: IEEE International conference on cyber technology in automation, control, and intelligent systems. IEEE, pp 115–120Google Scholar
  11. 11.
    Jin L, Hou H, Jiang Y (2012) Driver intention recognition based on continuous hidden Markov model. In: International conference on transportation, mechanical, and electrical engineering. IEEE, pp 739–742Google Scholar
  12. 12.
    Zheng Y, Hansen J H L (2017) Lane-change detection from steering signal using spectral segmentation and learning-based classification. IEEE Trans Intell Veh 2(1):14–24CrossRefGoogle Scholar
  13. 13.
    Ramyar S, Homaifar A, Karimoddini A et al. (2016) Identification of anomalies in lane change behavior using one-class SVM. In: IEEE International conference on systems, man, and cybernetics, pp 4405–4410Google Scholar
  14. 14.
    Hou Y, Edara P, Sun C (2014) Modeling mandatory lane changing using Bayes classifier and decision trees. IEEE Trans Intell Transport Syst 15(2):647–655CrossRefGoogle Scholar
  15. 15.
    Schubert R, Schulze K, Wanielik G (2010) Situation assessment for automatic lane-change maneuvers. IEEE Trans Intell Transport Syst 11(3):607–616CrossRefGoogle Scholar
  16. 16.
    Kasper D, Weidl G, Dang T et al. (2012) Object-oriented Bayesian networks for detection of lane change maneuvers. In: Intelligent vehicles symposium. IEEE, pp 673–678Google Scholar
  17. 17.
    Ulbrich S, Maurer M (2015) Situation assessment in tactical lane change behavior planning for automated vehicles. In: IEEE International conference on intelligent transportation systems, pp 975–981Google Scholar
  18. 18.
    Yan F, Eilers M, Ldtke A et al. (2016) Developing a model of driver’s uncertainty in lane change situations for trustworthy lane change decision aid systems. In: Intelligent vehicles symposium. IEEE, pp 406–411Google Scholar
  19. 19.
    Peng J, Guo Y, Fu R, Yuan W, Wang C (2015) Multiparameter prediction of drivers lane-changing behaviour with neural network model. Appl Ergon 50:207–217CrossRefGoogle Scholar
  20. 20.
    Tomar R S, Verma S (2013) Lane change trajectory prediction using artificial neural network. Int J Veh Safe 6(3):213–234CrossRefGoogle Scholar
  21. 21.
    Leonhardt V, Wanielik G (2017) Feature evaluation for lane change prediction based on driving situation and driver behavior. In: IEEE 20th International conference on information fusion, pp 1–7Google Scholar
  22. 22.
    Li J, Mei X, Prokhorov D et al. (2017) Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans Neural Netw Learn Syst 28(3):690–703CrossRefGoogle Scholar
  23. 23.
    Olabiyi O, Martinson E, Chintalapudi V et al. (2017) Driver action prediction using deep (bidirectional) recurrent neural network. arXiv:
  24. 24.
    lexandru G, Tejaswi K, Smita V et al. (2016) DeepLanes: End-to-end lane position estimation using deep neural networks. In: IEEE Conference on computer vision and pattern recognition, pp 38–45Google Scholar
  25. 25.
    Kin J, Lee M (2014) Robust lane detection based on convolutional neural network and random sample consensus. In: International Conference On Neural Information Processing, pp 454–461Google Scholar
  26. 26.
    Woo H et al. (2017) Lane-change detection based on vehicle-trajectory prediction. IEEE Robot Autom Lett 2(2):1109–1116CrossRefGoogle Scholar
  27. 27.
    Yao W, Zhao H, Bonnifait P, Zha H (2013) Lane change trajectory prediction by using recorded human driving data. In: Intelligent vehicles symposium. IEEE, pp 430–436Google Scholar
  28. 28.
    Nilsson J, Brannstrom M, Coelingh E et al. (2017) Lane change maneuvers for automated vehicles. IEEE Trans Intell Transport Syst 18(5):1087–1096CrossRefGoogle Scholar
  29. 29.
    Satzoda R K, Trivedi M M (2015) Drive analysis using vehicle dynamics and vision-based lane semantics. IEEE Trans Intell Transport Syst 16(1):9–18CrossRefGoogle Scholar
  30. 30.
    Xu G, Liu L, Ou Y et al. (2012) Dynamic modeling of driver control strategy of lane-change behavior and trajectory planning for collision prediction. IEEE Trans Intell Transport Syst 13(3):1138–1155CrossRefGoogle Scholar
  31. 31.
    Murphey Y, Kochhar D, Watta P, Wang X et al. (2015) Driver lane change prediction using physiological measures. SAE Int J Transport Safe. 3(2):118–125Google Scholar
  32. 32.
    Bach F R, Jordan M I (2004) Learning spectral clustering. Proc Adv Neural Inf Process Syst 16:305–312Google Scholar
  33. 33.
    Hajinoroozi M, Mao Z, Jung T P et al. (2016) EEG-based prediction of driver’s cognitive performance by deep convolutional neural network. Signal Process Image Commun 47:549–555CrossRefGoogle Scholar
  34. 34.
    Yang J B, Nguyen M N, San P P, Li X L, Krishnaswamy S h (2015) Deep convolutional neural networks on multichannel time series for human activity recognition. In: Proceedings of the international joint conference on artificial intelligence, pp 25–31Google Scholar
  35. 35.
    Malhotra P, Vig L, Shroff G, Agarwal P (2015) Long short term memory networks for anomaly detection in time series. In: Proceedings of European symposium on artificial neural networks, p 89Google Scholar
  36. 36.
    Wakasugi T (2005) A study on warning timing for lane change decision aid systems based on driver’s lane change maneuver. In: 19th International technical conference on the enhanced safety of vehiclesGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Logistics EngineeringWuhan University of TechnologyWuhanChina
  2. 2.Department of Electrical and Computer EngineeringUniversity of Michigan-DearbornDearbornUSA

Personalised recommendations