, Volume 101, Issue 12, pp 1837–1860 | Cite as

Personalized detection of lane changing behavior using multisensor data fusion

  • Jun GaoEmail author
  • Yi Lu Murphey
  • Honghui Zhu


Side swipe accidents occur primarily when drivers attempt an improper lane change, drift out of lane, or the vehicle loses lateral traction. In this paper, a fusion approach is introduced that utilizes multiple differing modality data, such as video data, GPS data, wheel odometry data, potentially IMU data collected from data logging device (DL1 MK3) for detecting driver’s behavior of lane changing by using a novel dimensionality reduction model, collaborative representation optimized projection classifier (CROPC). The criterion of CROPC is maximizing the collaborative representation based between-class scatter and minimizing the collaborative representation based within-class scatter in the transformed space simultaneously. For lane change detection, both feature-level fusion and decision-level fusion are considered. In the feature-level fusion, features generated from multiple differing modality data are merged before classification while in the decision-level fusion, an improved Dempster–Shafer theory based on correlation coefficient, DST-CC is presented to combine the classification outcomes from two classifiers, each corresponding to one kind of the data. The results indicate that the introduced fusion approach using a CROPC performs significantly better in terms of detection accuracy, in comparison to other state-of-the-art classifiers.


Lane change detection CROPC DST-CC Sensor fusion Machine learning 



This research is supported in part by Research Grants from Toyota Research Institute (TRI).


  1. 1.
    D’Agostino C, Saidi A, Scouarnec G, Chen L (2015) Learning-based driving events recognition and its application to digital roads. IEEE Trans Intell Trans Syst 16(4):2155–2166CrossRefGoogle Scholar
  2. 2.
    McCall J, Trivedi M (2016) Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans Intell Trans Syst 7(1):20–37CrossRefGoogle Scholar
  3. 3.
    Kasper D, Weidl G, Dang T et al (2012) Object-oriented Bayesian networks for detection of lane change maneuvers. In: Intelligent vehicles symposium. IEEE, pp 673–678Google Scholar
  4. 4.
    Huang AS, Moore D, Antone M, Olson E, Teller S (2009) Finding multiple lanes in urban road networks with vision and lidar. Auton Robot 26(2):103–122CrossRefGoogle Scholar
  5. 5.
    Gu X, Zang A, Huang X et al (2015) Fusion of color images and LiDAR data for lane classification. In: 23rd Sigspatial international conference on advances in geographic information systems. ACM, p 69Google Scholar
  6. 6.
    Tran D, Sheng W, Liu L et al (2015) A hidden Markov model based driver intention prediction system. In: International conference on cyber technology in automation, control, and intelligent systems. IEEE, pp 115–120Google Scholar
  7. 7.
    Jin L, Hou H, Jiang Y (2012) Driver intention recognition based on continuous hidden Markov model. In: International conference on transportation, mechanical, and electrical engineering. IEEE, pp 739–742Google Scholar
  8. 8.
    Zheng Y, Hansen John HL (2017) Lane-change detection from steering signal using spectral segmentation and learning-based classification. IEEE Trans Intell Veh 2(1):14–24CrossRefGoogle Scholar
  9. 9.
    Morris B, Doshi A, Trivedi M (2011) Lane change intent prediction for driver assistance: on-road design and evaluation. In: Intelligent vehicles symposium. IEEE, pp 895–901Google Scholar
  10. 10.
    Kim IH, Bong JH, Park J et al (2017) Prediction of driver’s intention of lane change by augmenting sensor information using machine learning techniques. Sensors 17(6):1350CrossRefGoogle Scholar
  11. 11.
    Ramyar S, Homaifar A, Karimoddini A, et al (2016) Identification of anomalies in lane change behavior using one-class SVM. In: International conference on systems, man, and cybernetics. IEEE, pp 4405–4410Google Scholar
  12. 12.
    Hou Y, Edara P, Sun C (2014) Modeling mandatory lane changing using bayes classifier and decision trees. IEEE Trans Intell Transp Syst 15(2):647–655CrossRefGoogle Scholar
  13. 13.
    Schubert R, Schulze K, Wanielik G (2010) Situation assessment for automatic lane-change maneuvers. IEEE Trans Intell Transp Syst 11(3):607–616CrossRefGoogle Scholar
  14. 14.
    Ulbrich S, Maurer M (2015) Situation assessment in tactical lane change behavior planning for automated vehicles. In: IEEE international conference on intelligent transportation systems, pp 975–981Google Scholar
  15. 15.
    Yan F, Eilers M, Ludtke A et al (2016) Developing a model of driver’s uncertainty in lane change situations for trustworthy lane change decision aid systems. In: Intelligent vehicles symposium. IEEE, pp 406–411Google Scholar
  16. 16.
    Peng J, Guo Y, Fu R, Yuan W, Wang C (2015) Multiparameter prediction of drivers’ lane-changing behaviour with neural network model. Appl Ergon 50:207–217CrossRefGoogle Scholar
  17. 17.
    Tomar RS, Verma S (2013) Lane change trajectory prediction using artificial neural network. Int J Veh Saf 6(3):213–234CrossRefGoogle Scholar
  18. 18.
    Leonhardt V, Wanielik G (2017) Feature evaluation for lane change prediction based on driving situation and driver behavior. In: IEEE 20th international conference on information fusion, pp 1–7Google Scholar
  19. 19.
    Li J, Mei X, Prokhorov D et al (2017) Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans Neural Netw Learn Syst 28(3):690–703CrossRefGoogle Scholar
  20. 20.
    Olabiyi O, Martinson E, Chintalapudi V, et al (2017) Driver action prediction using deep (bidirectional) recurrent neural network. arXiv:1706.02257
  21. 21.
    lexandru G, Tejaswi K, Smita V, et al (2016) DeepLanes: end-to-end lane position estimation using deep neural networks. In: IEEE conference on computer vision and pattern recognition, pp 38–45Google Scholar
  22. 22.
    Kin J, Lee M (2014) Robust lane detection based on convolutional neural network and random sample consensus. In: International conference on neural information processing. pp 454–461Google Scholar
  23. 23.
    Hanwool Woo et al (2017) Lane-change detection based on vehicle-trajectory prediction. IEEE Robot Autom Lett 2(2):1109–1116CrossRefGoogle Scholar
  24. 24.
    Yao W, Zhao H, Bonnifait P, Zha H (2013) Lane change trajectory prediction by using recorded human driving data. In: Intelligent vehicles symposium. IEEE, pp 430–436Google Scholar
  25. 25.
    Nilsson J, Brannstrom M, Coelingh E et al (2017) Lane change maneuvers for automated vehicles. IEEE Trans Intell Transp Syst 18(5):1087–1096CrossRefGoogle Scholar
  26. 26.
    Xu G, Liu L, Ou Y et al (2012) Dynamic modeling of driver control strategy of lane-change behavior and trajectory planning for collision prediction. IEEE Trans Intell Transp Syst 13(3):1138–1155CrossRefGoogle Scholar
  27. 27.
    Kim HT, Song B, Lee H et al (2015) Multiple vehicle recognition based on radar and vision sensor fusion for lane change assistance. J Inst Control 21(2):121–129Google Scholar
  28. 28.
    Cao G, Damerow F, Flade B et al (2016) Camera to map alignment for accurate low-cost lane-level scene interpretation. In: 19th international conference on intelligent transportation systems. IEEE, pp 498–504Google Scholar
  29. 29.
    Satzoda RK, Trivedi MM (2015) Drive analysis using vehicle dynamics and vision-based lane semantics. IEEE Trans Intell Transp Syst 16(1):9–18CrossRefGoogle Scholar
  30. 30.
    Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: which helps face recognition? In: IEEE international conference on computer vision, pp 471–478Google Scholar
  31. 31.
    Chen C, Li W, Tramel EW et al (2014) Reconstruction of hyperspectral imagery from random projections using multihypothesis prediction. IEEE Trans Geosci Remote Sens 52(1):365–374CrossRefGoogle Scholar
  32. 32.
    Shafer G (1976) A mathematical theory of evidence. Princeton University Press, PrincetonzbMATHGoogle Scholar
  33. 33.
    Jiang W (2016) A correlation coefficient of belief functions. arXiv:1612.05497
  34. 34.
  35. 35.
    Gao J, Murphey YL, Zhu HH (2018) Detection of lane-changing behavior using collaborative representation classifier-based sensor fusion. In: SAE international journal of transportation safety, vol 6, no. 2.
  36. 36.
    Chi Y, Porikli F (2012) Connecting the dots in multi-class classification: From nearest subspace to collaborative representation. In: IEEE international conference on computer vision and pattern recognition, pp 3602–3609Google Scholar
  37. 37.
    Cai S, Zhang L, Zuo W et al (2016) A probabilistic collaborative representation based approach for pattern classification. In: IEEE international conference on computer vision and pattern recognition, pp 2950–2959Google Scholar
  38. 38.
    Xu Y, Du Q, Li W et al (2017) Low-complexity multiple collaborative representations for hyperspectral image classification. In: International society for optics and photonics on high-performance computing in geoscience and remote sensing VII, vol 10430, p 1043003Google Scholar
  39. 39.
    Brown G, Pocock A, Zhao MJ et al (2012) Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J Mach Learn Res 13(1):27–66MathSciNetzbMATHGoogle Scholar
  40. 40.
    Lu CY, Huang DS (2013) Optimized projections for sparse representation based classification. Neurocomputing 113:213–219CrossRefGoogle Scholar
  41. 41.
    Gao Q, Huang Y, Zhang H et al (2015) Discriminative sparsity preserving projections for image recognition. Pattern Recognit 48(8):2543–2553CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Austria, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Logistics EngineeringWuhan University of TechnologyWuhanChina
  2. 2.Department of Electrical and Computer EngineeringUniversity of Michigan-DearbornDearbornUSA

Personalised recommendations