Advertisement

A Survey of Scene Understanding by Event Reasoning in Autonomous Driving

  • Jian-Ru Xue
  • Jian-Wu FangEmail author
  • Pu Zhang
Review
First Online:

Abstract

Realizing autonomy is a hot research topic for automatic vehicles in recent years. For a long time, most of the efforts to this goal concentrate on understanding the scenes surrounding the ego-vehicle (autonomous vehicle itself). By completing low-level vision tasks, such as detection, tracking and segmentation of the surrounding traffic participants, e.g., pedestrian, cyclists and vehicles, the scenes can be interpreted. However, for an autonomous vehicle, low-level vision tasks are largely insufficient to give help to comprehensive scene understanding. What are and how about the past, the on-going and the future of the scene participants? This deep question actually steers the vehicles towards truly full automation, just like human beings. Based on this thoughtfulness, this paper attempts to investigate the interpretation of traffic scene in autonomous driving from an event reasoning view. To reach this goal, we study the most relevant literatures and the state-of-the-arts on scene representation, event detection and intention prediction in autonomous driving. In addition, we also discuss the open challenges and problems in this field and endeavor to provide possible solutions.

Keywords

Autonomous vehicle scene understanding event reasoning intention prediction scene representation 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    M. M. Waldrop. Autonomous vehicles: No drivers required. Nature, vol.518, no.7537, pp.20–23, 2015. DOI: 10.1038/518020a.CrossRefGoogle Scholar
  2. [2]
    J. Mervis. Are We Going Too Fast on Driverless Cars? http://www.sciencemag.org/news/2017/12/are-wegoing-too-fast-driverless-cars, December 14, 2017.Google Scholar
  3. [3]
    Y. Y. Zheng, J. Yao. Multi-angle face detection based on DP-adaboost. International Journal of Automation and Computing, vol.12, no.4, pp.421–431, 2015. DOI: 10.1007/s11633-014-0872-8.CrossRefGoogle Scholar
  4. [4]
    H. G. Ren, W. M. Liu, T. Shi, F. J. Li. Compressive tracking based on online Hough forest. International Journal of Automation and Computing, vol. 14, no.4, pp.396–406, 2017. DOI: 10.1007/s11633-017-1083-x.CrossRefGoogle Scholar
  5. [5]
    J. W. Fang, H. K. Xu, Q. Wang, T. J. Wu. Online hash tracking with spatio-temporal saliency auxiliary. Computer Vision and Image Understanding, vol. 160, pp. 57–72, 2017. DOI: 10.1016/j.cviu.2017.03.006.CrossRefGoogle Scholar
  6. [6]
    S. Arumugadevi, V. Seenivasagam. Color image segmentation using feedforward neural networks with FCM. International Journal ofAutomation and Computing, vol. 13, no. 5, pp. 491–500, 2016. DOI: 10.1007/s11633-016-0975-5.CrossRefGoogle Scholar
  7. [7]
    J. F. Bonnefon, A. Shariff, I. Rahwan. The social dilemma of autonomous vehicles. Science, vol. 352, no. 6293, pp. 1573–1576, 2016. DOI: 10.1126/science.aaf2654.CrossRefGoogle Scholar
  8. [8]
    J. Janai, F. Guney, A. Behl, A. Geiger. Computer vision for autonomous vehicles: Problems, datasets and state-ofthe-art. arXiv:1704.05519, 2017.Google Scholar
  9. [9]
    J. R. Xue, D. Wang, S. Y. Du, D. X. Cui, Y. Huang, N. N. Zheng. A vision-centered multi-sensor fusing approach to self-localization and obstacle perception for robotic cars. Frontiers of Information Technology & Electronic Engineering, vol. 18, no. 1, pp. 122–138, 2017. DOI: 10.1631/FITEE. 1601873.CrossRefGoogle Scholar
  10. [10]
    H. Zhu, K. V. Yuen, L. Mihaylova, H. Leung. Overview of environment perception for intelligent vehicles. IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 10, pp. 2584–2601, 2017. DOI: 10.1109/TITS.2017.2658662.CrossRefGoogle Scholar
  11. [11]
    D. L. Waltz. Understanding scene descriptions as event simulations. In Proceedings of the 18th Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, Philadelphia, USA, pp.7–11, 1980. DOI: 10.3115/981436.981439.CrossRefGoogle Scholar
  12. [12]
    Y. Q. Hou, S. Hornauer, K. Zipser. Fast recurrent fully convolutional networks for direct perception in autonomous driving. arXiv:1711.06459, 2017.Google Scholar
  13. [13]
    H. Z. Xu, Y. Gao, F. Yu, T. Darrell. End-to-end learning of driving models from large-scale video datasets. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Honolulu, USA, pp.3530–3538, 2017. DOI: 10.1109/CVPR.2017.376.Google Scholar
  14. [14]
    T. Fernando, S. Denman, S. Sridharan, C. Fookes. Going deeper: Autonomous steering with neural memory networks. In Proceedings of IEEE International Conference on Computer Vision Workshop, IEEE, Venice, Italy, pp.214–221, 2017. DOI: 10.1109/ICCVW.2017.34.Google Scholar
  15. [15]
    C. Thorpe, M. H. Hebert, T. Kanade, S. A. Shafer. Vision and navigation for the Carnegie-Mellon Navlab. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, no. 3, pp. 362–372, 1988. DOI: 10.1109/34.3900.CrossRefGoogle Scholar
  16. [16]
    M. Buehler, K. Iagnemma, S. Singh. The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, Berlin, Heidelberg, Germany: Springer, 2009. DOI: 10.1007/9783-642-03991-1.CrossRefGoogle Scholar
  17. [17]
    J. Hooper. From DARPA Grand Challenge 2004DARPA’ s Debacle in The Desert. https://www.popsci.com/scitech/article/2004-06/darpagrand-challenge-2004darpas-debacle-desert, June 4, 2004.Google Scholar
  18. [18]
    S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron, J. Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann, K. Lau, C. Oakley, M. Palatucci, V. Pratt, P. Stang, S. Strohband, C. Dupont, L. E. Jendrossek, C. Koelen, C. Markey, C. Rummel, J. Van Niekerk, E. Jensen, P. Alessandrini, G. Bradski, B. Davies, S. Ettinger, A. Kaehle, A. Nefian, P. Mahoney. Stanley: The robot that won the DARPA grand challenge. The 2005 DARPA Grand Challenge, M. Buehler, K. Iagnemma, S. Singh, Eds., Berlin, Heidelberg, Germany: Springer, 2007. DOI: 10.1007/978-3-540-73429-1_1.Google Scholar
  19. [19]
    A. Geiger, P. Lenz, R. Urtasun. Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Providence, USA, pp. 33543361, 2012. DOI: 10.1109/CVPR.2012.6248074.Google Scholar
  20. [20]
    G. J. Brostow, J. Fauqueur, R. Cipolla. Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters, vol.30, no.2, pp.88, 2009. DOI: 10.1016/j.patrec.2008.04.005.CrossRefGoogle Scholar
  21. [21]
    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 3213–3223, 2016. DOI: 10.1109/CVPR.2016.350.Google Scholar
  22. [22]
    A. Gaidon, Q. Wang, Y. Cabon, E. Vig. Virtual worlds as proxy for multi-object tracking analysis. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4340–4349, 2016. DOI: 10.1109/CVPR.2016.470.Google Scholar
  23. [23]
    W. Maddern, G. Pascoe, C. Linegar, P. Newman. 1 year, 1000 km: The Oxford RobotCar dataset. International Journal of Robotics Research, vol. 36, no. 1, pp. 3–15, 2017. DOI: 10.1177/0278364916679498.CrossRefGoogle Scholar
  24. [24]
    J. V. Dueholm, M. S. Kristoffersen, R. K. Satzoda, E. Ohn-Bar, T. B. Moeslund, M. M. Trivedi. Multiperspective vehicle detection and tracking: Challenges, dataset, and metrics. In Proceedings of the 19th International Conference on Intelligent Transportation Systems, IEEE, Rio de Janeiro, Brazil, pp.959–964, 2016. DOI: 10.1109/ITSC.2016.7795671.Google Scholar
  25. [25]
    C. Wang, Y. K. Fang, H. J. Zhao, C. Z. Guo, S. Mita, H. B. Zha. Probabilistic inference for occluded and Multiview on-road vehicle detection. IEEE Transactions on Intelligent Transportation Systems, vol.17, no.1, pp.215–229, 2015. DOI: 10.1109/TITS.2015.2466109.CrossRefGoogle Scholar
  26. [26]
    D. Hoiem, S. K. Divvala, J. H. Hays. Pascal VOC 2008 challenge. World Literature Today, 2009.Google Scholar
  27. [27]
    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. H. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, F. F. Li. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015. DOI: 10.1007/s11263-015-0816-y.MathSciNetCrossRefGoogle Scholar
  28. [28]
    A. Milan, L. Leal-Taixe, I. Reid, S. Roth, K. Schindler. MOT16: A benchmark for multi-object tracking. arXiv:1603.00831, 2016.Google Scholar
  29. [29]
    F. C. Heilbron, V. Escorcia, B. Ghanem, J. C. Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 961–970, 2015. DOI: 10.1109/CVPR.2015.7298698.Google Scholar
  30. [30]
    T. Deng, K. F. Yang, Y. J. Li, H. M. Yan. Where does the driver look? Top-down-based saliency detection in a traffic driving environment. IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 7, pp. 2051–2062, 2016. DOI: 10.1109/TITS.2016.2535402.CrossRefGoogle Scholar
  31. [31]
    A. Geiger, M. Lauer, R. Urtasun. A generative model for 3D urban scene understanding from movable platforms. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Colorado Springs, USA, pp. 19451952, 2011. DOI: 10.1109/CVPR.2011.5995641.Google Scholar
  32. [32]
    J. M. Zhang, S. Sclaroff. Exploiting surroundedness for saliency detection: A Boolean map approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.38, no.5, pp.889, 2016. DOI: 10.1109/TPAMI.2015.2473844.CrossRefGoogle Scholar
  33. [33]
    L. Zhou, Y. F. Ju, J. W. Fang, J. R. Xue. Saliency detection via background invariance in scale space. Journal of Electronic Imaging, vol.26, no.4, Article number 043021, 2017. DOI: 10.1117/1.JEI.26.4.043021.Google Scholar
  34. [34]
    Q. Wang, Y. Yuan, P. K. Yan, X. L. Li. Saliency detection by multiple-instance learning. IEEE Transactions on Cybernetics, vol.43, no.2, pp.660–672, 2013. DOI: 10.1109/TSMCB.2012.2214210.CrossRefGoogle Scholar
  35. [35]
    S. F. He, R. W. H. Lau. Exemplar-driven top-down saliency detection via deep association. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 5723–5732, 2016. DOI: 10.1109/CVPR.2016.617.Google Scholar
  36. [36]
    J. M. Yang, M. H. Yang. Top-down visual saliency via joint CRF and dictionary learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 3, pp. 576–588, 2017. DOI: 10.1109/TPAMI.2016.2547384.CrossRefGoogle Scholar
  37. [37]
    J. T. Pan, E. Sayrol, X. Giro-I-Nieto, K. McGuinness, N. E. O’Connor. Shallow and deep convolutional networks for saliency prediction. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, NV, USA, pp.598–606, 2016. DOI: 10.1109/CVPR.2016.71.Google Scholar
  38. [38]
    Y. Xia, D. Q. Zhang, A. Pozdnoukhov, K. Nakayama, K. Zipser, D. Whitney. Training a network to attend like human drivers saves it from common but misleading loss functions. arXiv:1711.06406, 2017.Google Scholar
  39. [39]
    Y. Xie, L. F. Liu, C. H. Li, Y. Y. Qu. Unifying visual saliency with hog feature learning for traffic sign detection. In Proceedings of IEEE Intelligent Vehicles Symposium, IEEE, Xi’an, China, pp. 24–29, 2009. DOI: 10.1109/IVS.2009.5164247.Google Scholar
  40. [40]
    W. J. Won, M. Lee, J. W. Son. Implementation of road traffic signs detection based on saliency map model. In Proceedings of IEEE Intelligent Vehicles Symposium, IEEE, Eindhoven, Netherlands, pp. 542–547, 2008. DOI: 10.1109/IVS.2008.4621144.Google Scholar
  41. [41]
    D. D. Wang, X. W. Hou, J. W. Xu, S. G. Yue, C. L. Liu. Traffic sign detection using a cascade method with fast feature extraction and saliency test. IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 12, pp. 32903302, 2017. DOI: 10.1109/TITS.2017.2682181.Google Scholar
  42. [42]
    J. Kim, S. Kim, R. Mallipeddi, G. Jang, M. Lee. Adaptive driver assistance system based on traffic information saliency map. In Proceedings of International Joint Conference on Neural Networks, IEEE, Vancouver, Canada, pp. 1918–1923, 2016. DOI: 10.1109/IJCNN.2016.7727434.Google Scholar
  43. [43]
    V. John, K. Yoneda, Z. Liu, S. Mita. Saliency map generation by the convolutional neural network for real-time traffic light detection using template matching. IEEE Transactions on Computational Imaging, vol. 1, no. 3, pp. 159–173, 2015. DOI: 10.1109/TCI.2015.2480006.MathSciNetCrossRefGoogle Scholar
  44. [44]
    H. L. Kuang, K. F. Yang, L. Chen, Y. J. Li, L. L. H. Chan, H. Yan. Bayes saliency-based object proposal generator for nighttime traffic images. IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 3, pp. 814–825, 2017. DOI: 10.1109/TITS.2017.2702665.CrossRefGoogle Scholar
  45. [45]
    R. Timofte, K. Zimmermann, L. V. Gool. Multi-view traffic sign detection, recognition, and 3D localisation. Machine Vision and Applications, vol.25, no.3, pp.633–647, 2014. DOI: 10.1007/s00138-011-0391-3.CrossRefGoogle Scholar
  46. [46]
    S. Alletto, A. Palazzi, F. Solera, S. Calderara, R. Cucchiara. DR(eye)VE: A dataset for attention-based tasks with applications to autonomous and assisted driving. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE, LasVegas, USA, 2016. DOI: 10.1109/CVPRW.2016.14.Google Scholar
  47. [47]
    A. Palazzi, F. Solera, S. Calderara, S. Alletto, R. Cucchiara. Where should you attend while driving? arXiv:1611.08215, 2016.Google Scholar
  48. [48]
    C. Landsiedel, D. Wollherr. Road geometry estimation for urban semantic maps using open data. Advanced Robotics, vol.31, no.5, pp.282–290, 2017. DOI: 10.1080/01691864.2016.1250675.CrossRefGoogle Scholar
  49. [49]
    E. Levinkov, M. Fritz. Sequential Bayesian model update under structured scene prior for semantic road scenes labeling. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Sydney, Australia, pp. 1321–1328, 2013. DOI: 10.1109/ICCV.2013.167.Google Scholar
  50. [50]
    Z. Y. Zhang, S. Fidler, R. Urtasun. Instance-level segmentation for autonomous driving with deep densely connected MRFs. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 669–677, 2016. DOI: 10.1109/CVPR.2016.79.Google Scholar
  51. [51]
    T. Cavallari, Semantic Slam: A New Paradigm for Object Recognition and Scene Reconstruction, Ph. D. dissertation, University of Bologna, Italy, 2017.Google Scholar
  52. [52]
    S. C. Zhou, R. Yan, J. X. Li, Y. K. Chen, H. J. Tang. A brain-inspired SLAM system based on ORB features. International Journal of Automation and Computing, vol. 14, no.5, pp. 564–575, 2017. DOI: 10.1007/s11633-017-1090-y.CrossRefGoogle Scholar
  53. [53]
    B. Zhao, J. S. Feng, X. Wu, S. C. Yan. A survey on deep learning-based fine-grained object classification and semantic segmentation. International Journal of Automation and Computing, vol. 14, no. 2, pp. 119–135, 2017. DOI: 10.1007/s11633-017-1053-3.CrossRefGoogle Scholar
  54. [54]
    H. Kong, J. Y. Audibert, J. Ponce. Vanishing point detection for road detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Miami, USA, pp. 96–103, 2009. DOI: 10.1109/CVPR.2009.5206787.Google Scholar
  55. [55]
    H. Kong, S. E. Sarma, F. Tang. Generalizing Laplacian of Gaussian filters for vanishing-point detection. IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 1, pp. 408–418, 2013. DOI: 10.1109/TITS.2012.2216878.CrossRefGoogle Scholar
  56. [56]
    J. J. Shi, J. X. Wang, F. F. Fu. Fast and robust vanishing point detection for unstructured road following. IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 4, pp. 970–979, 2016. DOI: 10.1109/TITS.2015.2490556.CrossRefGoogle Scholar
  57. [57]
    J. M. Alvarez, T. Gevers, A. M. Lopez. 3D scene priors for road detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, San Francisco, USA, pp.57–64, 2010. DOI: 10.1109/CVPR.2010.5540228.Google Scholar
  58. [58]
    E. Casapietra, T. H. Weisswange, C. Goerick, F. Kummert. Enriching a spatial road representation with lanes and driving directions. In Proceedings of the 19th International Conference on Intelligent Transportation Systems, IEEE, Rio de Janeiro, Brazil, pp. 1579–1585, 2016. DOI: 10.1109/ITSC.2016.7795768.Google Scholar
  59. [59]
    A. Seff, J. X. Xiao. Learning from maps: Visual common sense for autonomous driving. arXiv:1611.08583, 2016.Google Scholar
  60. [60]
    M. Y. Liu, S. X. Lin, S. Ramalingam, O. Tuzel. Layered interpretation of street view images. arXiv:1506.04723, 2015.CrossRefGoogle Scholar
  61. [61]
    A. Geiger, M. Lauer, C. Wojek, C. Stiller, R. Urtasun. 3D traffic scene understanding from movable platforms. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 5, pp. 1012–1025, 2014. DOI: 10.1109/TPAMI.2013.185.CrossRefGoogle Scholar
  62. [62]
    A. Ess, T. Mueller, H. Grabner, L. Van Gool. Segmentationbased urban traffic scene understanding. In Proceedings of British Machine Vision Conference, London, UK, 2009. DOI: 10.5244/C.23.84.Google Scholar
  63. [63]
    B. Kitt, A. Geiger, H. Lategahn. Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme. In Proceedings of IEEE Intelligent Vehicles Symposium, IEEE, San Diego, USA, pp. 486–492, 2010. DOI: 10.1109/IVS.2010.5548123.Google Scholar
  64. [64]
    S. Thrun, W. Burgard, D. Fox. Probabilistic Robotics (Intelligent Robotics and Autonomous Agents), Cambridge, Mass, UK: MIT, 2005.zbMATHGoogle Scholar
  65. [65]
    H. Y. Zhang, A. Geiger, R. Urtasun. Understanding high-level semantics by modeling traffic patterns. In Proceedings of IEEE Conference on Computer Vision, IEEE, Sydney, Australia, pp. 3056–3063, 2013. DOI: 10.1109/ICCV.2013.379.Google Scholar
  66. [66]
    C. Y. Chen, A. Seff, A. Kornhauser, J. X. Xiao. Deep-Driving: Learning affordance for direct perception in autonomous driving. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Santiago, Chile, pp.2722–2730, 2015. DOI: 10.1109/ICCV.2015.312.Google Scholar
  67. [67]
    P. Stahl, B. Donmez, G. A. Jamieson. Anticipation in driving: The role of experience in the efficacy of pre-event conflict cues. IEEE Transactions on Human-Machine Systems, vol.44, no.5, pp.603–613, 2014. DOI: 10.1109/THMS.2014.2325558.CrossRefGoogle Scholar
  68. [68]
    S. J. Pan, Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, 2010. DOI: 10.1109/TKDE.2009.191.CrossRefGoogle Scholar
  69. [69]
    N. Segev, M. Harel, S. Mannor, K. Crammer, R. El-Yaniv. Learn on source, refine on target: A model transfer learning framework with random forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 9, pp.1811–1824, 2017. DOI: 10.1109/TPAMI.2016.2618118.CrossRefGoogle Scholar
  70. [70]
    D. Mitrovic. Reliable method for driving events recognition. IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 2, pp. 198–205, 2005. DOI: 10.1109/TITS.2005.848367.CrossRefGoogle Scholar
  71. [71]
    B. F. Wu, Y. H. Chen, C. H. Yeh, Y. F. Li. Reasoningbased framework for driving safety monitoring using driving event recognition. IEEE Transactions on Intelligent Transportation Systems, vol.14, no. 3, pp. 1231–1241, 2013. DOI: 10.1109/TITS.2013.2257759.Google Scholar
  72. [72]
    A. Ramirez, E. Ohn-Bar, M. Trivedi. Integrating motion and appearance for overtaking vehicle detection. In Proceedings of IEEE Intelligent Vehicles Symposium Proceedings, IEEE, Dearborn, USA, pp.96–101, 2014. DOI: 10.1109/IVS.2014.6856598.Google Scholar
  73. [73]
    J. D. Alonso, E. R. Vidal, A. Rotter, M. Muhlenberg. Lane-change decision aid system based on motiondriven vehicle tracking. IEEE Transactions on Vehicular Technology, vol. 57, no. 5, pp. 2736–2746, 2008. DOI: 10.1109/TVT.2008.917220.CrossRefGoogle Scholar
  74. [74]
    Y. Zhu, D. Comaniciu, M. Pellkofer, T. Koehler. Reliable detection of overtaking vehicles using robust information fusion. IEEE Transactions on Intelligent Transportation Systems, vol.7, no.4, pp.401–414, 2006. DOI: 10.1109/TITS.2006.883936.CrossRefGoogle Scholar
  75. [75]
    F. Garcia, P. Cerri, A. Broggi, A. De La Escalera, J. M. Armingol. Data fusion for overtaking vehicle detection based on radar and optical flow. In Proceedings of IEEE Intelligent Vehicles Symposium, IEEE, Alcala de Henares, Spain, pp. 494–499, 2012. DOI: 10.1109/IVS.2012.6232199.Google Scholar
  76. [76]
    Deutscher Verkehrssicherheitsrat. DVR-Report: Fachmagazin für Verkehrssicherheit. https://www. dvr.de/presse/dvr-report/2017-04.Google Scholar
  77. [77]
    Auto Club Europa (ACE). Reviere der blinkmuffel. http: //www.ace-online.de/fileadmin/user_uploads/Der_Club/Dokumente/10.07.2008_Grafik_Blinkmuffel_l.pdf.Google Scholar
  78. [78]
    D. Kasper, G. Weidl, T. Dang, G. Breuel, A. Tamke, A. Wedel, W. Rosenstiel. Object-oriented Bayesian networks for detection of lane change maneuvers. IEEE Intelligent Transportation Systems Magazine, vol.4, no.3, pp.19–31, 2012. DOI: 10.1109/MITS.2012.2203229.CrossRefGoogle Scholar
  79. [79]
    W. Yao, Q. Q. Zeng, Y. P. Lin, D. H. Xu, H. J. Zhao, F. Guillemard, S. Geronimi, F. Aioun. On-road vehicle trajectory collection and scene-based lane change analysis: Part II. IEEE Transactions on Intelligent Transportation Systems, vol.18, no.1, pp.206–220, 2017. DOI: 10.1109/TITS.2016.2571724.CrossRefGoogle Scholar
  80. [80]
    T. Gindele, S. Brechtel, R. Dillmann. A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments. In Proceedings of the 13th International Conference on Intelligent Transportation Systems, IEEE, Funchal, Portugal, pp. 1625–1631, 2010. DOI: 10.1109/ITSC.2010.5625262.CrossRefGoogle Scholar
  81. [81]
    S. Sivaraman, B. Morris, M. Trivedi. Learning multi-lane trajectories using vehicle-based vision. In Proceedings of IEEE Conference on Computer Vision Workshops, IEEE, Barcelona, Spain, pp. 2070–2076, 2011. DOI: 10.1109/ICCVW. 2011.6130503.Google Scholar
  82. [82]
    R. K. Satzoda, M. M. Trivedi. Overtaking & receding vehicle detection for driver assistance and naturalistic driving studies. In Proceedings of the 17th International Conference on Intelligent Transportation Systems, IEEE, Qingdao, China, pp.697–702, 2014. DOI: 10.1109/ITSC.2014.6957771.Google Scholar
  83. [83]
    M. S. Kristoffersen, J. V. Dueholm, R. K. Satzoda, M. M. Trivedi, A. Mogelmose, T. B. Moeslund. Towards semantic understanding of surrounding vehicular maneuvers: A panoramic vision-based framework for realworld highway studies. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE, Las Vegas, USA, pp. 1584–1591, 2016. DOI: 10.1109/CVPRW.2016.197.Google Scholar
  84. [84]
    J. V. Dueholm, M. S. Kristoffersen, R. K. Satzoda, T. B. Moeslund, M. M. Trivedi. Trajectories and maneuvers of surrounding vehicles with panoramic camera arrays. IEEE Transactions on Intelligent Vehicles, vol. 1, no. 2, pp. 203214, 2016. DOI: 10.1109/TIV.2016.2622921.CrossRefGoogle Scholar
  85. [85]
    A. Khosroshahi, E. Ohn-Bar, M. M. Trivedi. Surround vehicles trajectory analysis with recurrent neural networks. In Proceedings of the 19th International Conference on Intelligent Transportation Systems, IEEE, Rio de Janeiro, Brazil, pp. 2267–2272, 2016. DOI: 10.1109/ITSC.2016.7795922.Google Scholar
  86. [86]
    S. Ernst, J. Rieken, M. Maurer. Behaviour recognition of traffic participants by using manoeuvre primitives for automated vehicles in urban traffic. In Proceedings of the 19th International Conference on Intelligent Transportation Systems, IEEE, Rio de Janeiro, Brazil, 2016. DOI: 10.1109/ITSC.2016.7795674.Google Scholar
  87. [87]
    S. Busch, T. Schindler, T. Klinger, C. Brenner. Analysis of Spatio-temporal traffic patterns based on pedestrian trajectories. In Proceedings of International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Prague, Czech Republic, vol.XLI-B2, pp.497–503, 2016. DOI: 10.5194/isprsarchives-XLI-B2-497-2016.CrossRefGoogle Scholar
  88. [88]
    J. Hariyono, K. H. Jo. Detection of pedestrian crossing road: A study on pedestrian pose recognition. Neurocomputing, vol.234, pp.144–153, 2017. DOI: 10.1016/j.neucom.2016.12.050.Google Scholar
  89. [89]
    R. M. Mueid, C. Ahmed, M. A. R. Ahad. Pedestrian activity classification using patterns of motion and histogram of oriented gradient. Journal on Multimodal User Interfaces, vol. 10, no. 4, pp. 299–305, 2016. DOI: 10.1007/s12193-015-0178-3.CrossRefGoogle Scholar
  90. [90]
    R. Quintero, I. Parra, D. F. Llorca, M. A. Sotelo. Pedestrian intention and pose prediction through dynamical models and behaviour classification. In Proceedings of the 18th International Conference on Intelligent Transportation Systems, IEEE, Las Palmas, Spain, pp. 83–88, 2015. DOI: 10.1109/ITSC.2015.22.Google Scholar
  91. [91]
    M. Ogawa, H. Fukamachi, R. Funayama, T. Kindo. CYKLS: Detect pedestrian’s dart focusing on an appearance change. In Proceedings of the 12th International Conference on Computer Vision, Springer-Verlag, Florence, Italy, pp. 556–565, 2012. DOI: 10.1007/978-3-642-33868-7_55.Google Scholar
  92. [92]
    F. H. Chan, Y. T. Chen, Y. Xiang, M. Sun. Anticipating accidents in dashcam videos. In Proceedings of the 13th Asian Conference on Computer Vision, Springer, Taipei, China, pp. 136–153, 2016. DOI: 10.1007/978-3-319-54190-7_9.Google Scholar
  93. [93]
    Y. J. Xia, W. W. Xu, L. M. Zhang, X. M. Shi, K. Mao. Integrating 3D structure into traffic scene understanding with RGB-D data. Neurocomputing, vol. 151, pp. 700–709, 2015. DOI: 10.1016/j.neucom.2014.05.091.CrossRefGoogle Scholar
  94. [94]
    A. Tageldin, M. H. Zaki, T. Sayed. Examining pedestrian evasive actions as a potential indicator for traffic conflicts. IET Intelligent Transport Systems, vol.11, no.5, pp.282–289, 2017. DOI: 10.1049/iet-its.2016.0066.CrossRefGoogle Scholar
  95. [95]
    F. Westerhuis, D. De Waard. Reading cyclist intentions: Can a lead cyclists behaviour be predicted? Accident Analysis & Prevention, vol. 105, pp. 146–155, 2017. DOI: 10.1016/j.aap.2016.06.026.CrossRefGoogle Scholar
  96. [96]
    D. Manstetten. Behaviour prediction and intention detection in UR:BAN VIE -overview and introduction. URBAN Human Factors in Traffic, K. Bengler, J. Druke, S. Hoffmann, D. Manstetten, A. Neukum, Eds., Wiesbaden, Germany: Springer, 2018. DOI: 10.1007/978-3-658-15418-9_8.Google Scholar
  97. [97]
    F. Schneemann, P. Heinemann. Context-based detection of pedestrian crossing intention for autonomous driving in urban environments. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Daejeon, South Korea, pp. 2243–2248, 2016. DOI: 10.1109/IROS.2016.7759351.Google Scholar
  98. [98]
    T. Fugger, B. Randles, A. Stein, W. Whiting, B. Gallagher. Analysis of pedestrian gait and Perception-reaction at signal-controlled crosswalk intersections. Transportation Research Record: Journal of the Transportation Research Board, vol. 1705, no. 1, pp.20–25, 2000. DOI: 10.3141/170504.Google Scholar
  99. [99]
    N. Schneider, D. M. Gavrila. Pedestrian path prediction with recursive Bayesian filters: A comparative study. In Proceedings of the 35th German Conference on Pattern Recognition, Springer, Saarbrücken, Germany, pp. 174–183, 2013. DOI: 10.1007/978-3-642-40602-7_18.Google Scholar
  100. [100]
    M. Goldhammer, M. Gerhard, S. Zernetsch, K. Doll, U. Brunsmann. Early prediction of a pedestrian’s trajectory at intersections. In Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems, IEEE, The Hague, The Netherlands, pp.237–242, 2013. DOI: 10.1109/ITSC.2013.6728239.Google Scholar
  101. [101]
    M. Goldhammer, K. Doll, U. Brunsmann, A. Gensler, B. Sick. Pedestrians trajectory forecast in public traffic with artificial neural networks. In Proceedings of the 22nd International Conference on Pattern Recognition, IEEE, Stockholm, Sweden, pp.4110–4115, 2014. DOI: 10.1109/ICPR.2014.704.Google Scholar
  102. [102]
    C. G. Keller, D. M. Gavrila. Will the pedestrian cross? A study on pedestrian path prediction. IEEE Transactions on Intelligent Transportation Systems, vol.15, no.2, pp.494–506, 2014. DOI: 10.1109/TITS.2013.2280766.CrossRefGoogle Scholar
  103. [103]
    S. Koehler, M. Goldhammer, S. Bauer, S. Zecha, K. Doll, U. Brunsmann, K. Dietmayer. Stationary detection of the Pedestrians intention at intersections. IEEE Intelligent Transportation Systems Magazine, vol. 5, no. 4, pp. 87–99, 2013. DOI: 10.1109/MITS.2013.2276939.CrossRefGoogle Scholar
  104. [104]
    J. F. P. Kooij, N. Schneider, F. Flohr, D. M. Gavrila. Context-based pedestrian path prediction. In Proceedings of the 13th European Conference on Computer Vision, Springer, Zurich, Switzerland, pp.618–633, 2014. DOI: 10.1007/978-3-319-10599-4_40.Google Scholar
  105. [105]
    J. Y. Kwak, B. C. Ko, J. Y. Nam. Pedestrian intention prediction based on dynamic fuzzy automata for vehicle driving at nighttime. Infrared Physics & Technology, vol.81, pp. 41–51, 2017. DOI: 10.1016/j.infrared.2016.12.014.Google Scholar
  106. [106]
    G. Q. Xu, L. Liu, Y. S. Ou, Z. J. Song. Dynamic modeling of driver control strategy of lane-change behavior and trajectory planning for collision prediction. IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 3, pp. 1138–1155, 2012. DOI: 10.1109/TITS.2012.2187447.CrossRefGoogle Scholar
  107. [107]
    R. N. Dang, J. Q. Wang, S. E. Li, K. Q. Li. Coordinated adaptive cruise control system with lane-change assistance. IEEE Transactions on Intelligent Transportation Systems, vol.16, no.5, pp.2373–2383, 2015. DOI: 10.1109/TITS.2015.2389527.CrossRefGoogle Scholar
  108. [108]
    W. Liu, S. W. Kim, K. Marczuk, M. H. Ang. Vehicle motion intention reasoning using cooperative perception on urban road. In Proceedings of the 17th International Conference on Intelligent Transportation Systems, IEEE, Qingdao, China, pp.424–430, 2014. DOI: 10.1109/ITSC.2014.6957727.Google Scholar
  109. [109]
    Y. Hou, P. Edara, C. Sun. Modeling mandatory lane changing using Bayes classifier and decision trees. IEEE Transactions on Intelligent Transportation Systems, vol. 15, no.2, pp. 647–655, 2014. DOI: 10.1109/TITS.2013.2285337.CrossRefGoogle Scholar
  110. [110]
    D. Lee, A. Hansen, J. K. Hedrick. Probabilistic inference of traffic participants lane change intention for enhancing adaptive cruise control. In Proceedings of IEEE Intelligent Vehicles Symposium, IEEE, Los Angeles, USA, pp.855–860, 2017. DOI: 10.1109/IVS.2017.7995823.Google Scholar
  111. [111]
    Y. L. Gu, Y. Hashimoto, L. T. Hsu, S. Kamijo. Motion planning based on learning models of pedestrian and driver behaviors. In Proceedings of the 19th International Conference on Intelligent Transportation Systems, IEEE, Rio de Janeiro, Brazil, pp.808–813, 2016. DOI: 10.1109/ITSC.2016.7795648.Google Scholar
  112. [112]
    W. D. Xu, J. Pan, J. Q. Wei, J. M. Dolan. Motion planning under uncertainty for on-road autonomous driving. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Hong Kong, China, pp. 2507–2512, 2014. DOI: 10.1109/ICRA.2014.6907209.Google Scholar
  113. [113]
    T. Y. Gu, J. M. Dolan, J. W. Lee. Automated tactical maneuver discovery, reasoning and trajectory planning for autonomous driving. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Daejeon, South Korea, pp.5474–5480, 2016. DOI: 10.1109/IROS.2016.7759805.Google Scholar
  114. [114]
    N. Nagasaka, M. Harada. Towards safe, smooth, and stable path planning for on-road autonomous driving under uncertainty. In Proceedings of the 19th International Conference on Intelligent Transportation Systems, IEEE, Rio de Janeiro, Brazil, pp.795–801, 2016. DOI: 10.1109/ITSC.2016.7795646.Google Scholar
  115. [115]
    K. Jo, M. Lee, J. Kim, M. Sunwoo. Tracking and behavior reasoning of moving vehicles based on roadway geometry constraints. IEEE Transactions on Intelligent Transportation Systems, vol.18, no.2, pp.460–476, 2017. DOI: 10.1109/TITS.2016.2605163.CrossRefGoogle Scholar
  116. [116]
    E. A. I. Pool, J. F. P. Kooij, D. M. Gavrila. Using road topology to improve cyclist path prediction. In Proceedings of IEEE Intelligent Vehicles Symposium, IEEE, Los Angeles, USA, pp. 289–296, 2017. DOI: 10.1109/IVS.2017.7995734.Google Scholar
  117. [117]
    N. Evestedt, E. Ward, J. Folkesson, D. Axehill. Interaction aware trajectory planning for merge scenarios in congested traffic situations. In Proceedings of the 19th International Conference on Intelligent Transportation Systems, IEEE, Rio de Janeiro, Brazil, pp. 465–472, 2016. DOI: 10.1109/ITSC.2016.7795596.Google Scholar
  118. [118]
    H. M. Eraqi, M. N. Moustafa, J. Honer. End-to-end deep learning for steering autonomous vehicles considering temporal dependencies. arXiv:1710.03804, 2017.Google Scholar
  119. [119]
    L. Caltagirone, M. Bellone, L. Svensson, M. Wahde. Simultaneous perception and path generation using fully convolutional neural networks. arXiv:1703.08987, 2017.Google Scholar
  120. [120]
    Z. Bylinskii, T. Judd, A. Oliva, A. Torralba, F. Durand. What do different evaluation metrics tell us about saliency models? arXiv:1604.03605, 2016.Google Scholar
  121. [121]
    B. W. Tatler. The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. Journal of Vision, vol. 7, no. 14–17, pp. 4.1–17, 2007. DOI: 10.1167/7.14.4.Google Scholar
  122. [122]
    R. J. Peters, A. Iyer, L. Itti, C. Koch. Components of bottom-up gaze allocation in natural images. Vision Research, vol. 45, no. 18, pp. 2397–2416, 2005. DOI: 10.1016/j.visres.2005.03.019.CrossRefGoogle Scholar
  123. [123]
    M. Kümmerer, T. S. A. Wallis, M. Bethge. Informationtheoretic model comparison unifies saliency metrics. In Proceedings of the National Academy of Sciences of the United States of America, vol. 112, no.52, pp. 16054–16059, 2015. DOI: 10.1073/pnas. 1510393112.CrossRefGoogle Scholar
  124. [124]
    M. J. Swain, D. H. Ballard. Color indexing. International Journal of Computer Vision, vol.7, no.1, pp.11–32, 1991. DOI: 10.1007/BF00130487.CrossRefGoogle Scholar
  125. [125]
    O. Le Meur, P. Le Callet, D. Barba. Predicting visual fixations on video based on low-level visual features. Vision Research, vol.47, no.19, pp.2483, 2498. DOI: 10.1016/j.visres.2007.06.015.CrossRefGoogle Scholar
  126. [126]
    O. Pele, M. Werman. A linear time histogram metric for improved sift matching. In Proceedings of the 10th European Conference on Computer Vision: Part III, Marseille, France, pp.495–508, 2008. DOI: 10.1007/978-3-540-88690-7_37.Google Scholar
  127. [127]
    Y. Rubner, C. Tomasi, L. J. Guibas. The earth movers distance as a metric for image retrieval. International Journal of Computer Vision, vol.40, no.2, pp.99–121, 2000. DOI: 10.1023/A:1026543900054.zbMATHCrossRefGoogle Scholar
  128. [128]
    C. Sammut, G. I. Webb. Encyclopedia of Machine Learning, Boston, MA: Springer, 2010.zbMATHCrossRefGoogle Scholar

Copyright information

© Institute of Automation, Chinese Academy of Sciences and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of Artificial Intelligence and RoboticsXi’an Jiaotong UniversityXi’anChina
  2. 2.Chang’an UniversityXi’anChina

Personalised recommendations

Cite article

Buy options