Autonomous driving: cognitive construction and situation understanding

Science China Information Sciences volume 62, Article number: 81101 (2019) Cite this article

Abstract

Autonomous vehicle is a kind of typical complex artificial intelligence system. In current research of autonomous driving, the most widely adopted technique is to use a basic framework of serial information processing and computations, which consists of four modules: perception, planning, decision-making, and control. However, this framework based on data-driven computing performs low computational efficiency, poor environmental understanding and self-learning ability. A neglected problem has long been how to understand and process environmental perception data from the sensors referring to the cognitive psychology level of the human driving process. The key to solving this problem is to construct a computing model with selective attention and self-learning ability for autonomous driving, which is supposed to possess the mechanism of memorizing, inferring and experiential updating, enabling it to cope with traffic scenarios with high noise, dynamic, and randomness. In addition, for the process of understanding traffic scenes, the efficiency of event-related mechanism is more significant than single-attribute scenario perception data. Therefore, an effective self-driving method should not be confined to the traditional computing framework of ‘perception, planning, decision-making, and control’. It is necessary to explore a basic computing framework that conforms to human driver’s attention, reasoning, learning, and decision-making mechanism with regard to traffic scenarios and build an autonomous system inspired by biological intelligence. In this article, we review the basic methods and main progress in current data-driven autonomous driving technologies, deeply analyze the limitations and major problems faced by related algorithms. Then, combined with authors’ research, this study discusses how to implement a basic cognitive computing framework of self-driving with selective attention and an event-driven mechanism from the basic viewpoint of cognitive science. It further describes how to use multi-sensor and graph data with semantic information (such as traffic maps and a spatial correlation of events) to realize the associative representations of objects and drivable areas, as well as the intuitive reasoning method applied to understanding the situations in different traffic scenarios. The computing framework of autonomous driving based on a selective attention mechanism and intuitive reasoning discussed in this study can adapt to a more complex, open, and dynamic traffic environment.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

References

  1. 1

    Thrun S, Montemerlo M, Dahlkamp H, et al. Stanley: the robot that won the DARPA grand challenge. J Field Robot, 2006, 23: 661–692

    Article  Google Scholar 

  2. 2

    Miller G A. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev, 1956, 63: 81–97

    Article  Google Scholar 

  3. 3

    Kahneman D, Treisman A, Gibbs B J. The reviewing of object files: object-specific integration of information. Cogn Psychol, 1992, 24: 175–219

    Article  Google Scholar 

  4. 4

    Kahneman D, Frederick S. Representativeness revisited: attribute substitution in intuitive judgment. In: Heuristics and Biases: the Psychology of Intuitive Judgment. New York: Cambridge University Press, 2002. 49–81

    Google Scholar 

  5. 5

    Jang Y, Song Y, Yu Y, et al. TGIF-QA: toward spatio-temporal reasoning in visual question answering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, 2017. 2758–2766

  6. 6

    Ojala T, Pietikainen M, Harwood D. Performance evaluation of texture measures with classification based on kullback discrimination of distributions. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, Jerusalem, 1994. 582–585

  7. 7

    Ojala T, Pietikainen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recogn, 1996, 29: 51–59

    Article  Google Scholar 

  8. 8

    Zhao L, Thorpe C E. Stereo- and neural network-based pedestrian detection. IEEE Trans Intell Transp Syst, 2000, 1: 148–154

    Article  Google Scholar 

  9. 9

    Yuan Y, Xiong Z T, Wang Q. An incremental framework for video-based traffic sign detection, tracking, and recognition. IEEE Trans Intell Transp Syst, 2017, 18: 1918–1929

    Article  Google Scholar 

  10. 10

    Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014. 580–587

  11. 11

    Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, 2015. 1440–1448

  12. 12

    Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, 2015. 91–99

  13. 13

    Liu W, Anguelov D, Erhan D, et al. Ssd: single shot multibox detector. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2016. 21–37

    Google Scholar 

  14. 14

    Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 779–788

  15. 15

    Redmon J, Farhadi A. Yolo9000: better, faster, stronger. 2017. ArXiv: 1612.08242

  16. 16

    Wu B C, Iandola F, Jin P H, et al. Squeezedet: unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, 2017

  17. 17

    Chen X Z, Ma H M, Wan J, et al. Multi-view 3D object detection network for autonomous driving. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 3

  18. 18

    Ku J, Mozifian M, Lee J, et al. Joint 3D proposal generation and object detection from view aggregation. 2017. ArXiv: 1712.02294

  19. 19

    Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? the kitti vision benchmark suite. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012. 3354–3361

  20. 20

    Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 3431–3440

  21. 21

    Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional encoder-decoder architecture for image segmentation. 2015. ArXiv: 1511.00561

  22. 22

    Chen L-C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected crfs. 2014. ArXiv: 1412.7062

  23. 23

    Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell, 2018, 40: 834–848

    Article  Google Scholar 

  24. 24

    Wang Q, Gao J Y, Yuan Y. A joint convolutional neural networks and context transfer for street scenes labeling. IEEE Trans Intell Transp Syst, 2018, 19: 1457–1470

    Article  Google Scholar 

  25. 25

    Oliveira G L, Burgard W, Brox T. Efficient deep models for monocular road segmentation. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016. 4885–4891

  26. 26

    Dai J F, He K M, Sun J. Instance-aware semantic segmentation via multi-task network cascades. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 3150–3158

  27. 27

    He K M, Gkioxari G, Dollar P, et al. Mask r-cnn. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), 2017. 2980–2988

  28. 28

    Li Y, Qi H Z, Dai J F, et al. Fully convolutional instance-aware semantic segmentation. 2016. ArXiv: 1611.07709

  29. 29

    Bosse M, Zlot R. Continuous 3D scan-matching with a spinning 2D laser. In: Proceedings of IEEE International Conference on Robotics and Automation, 2009. 4312–4319

  30. 30

    Baldwin I, Newman P. Laser-only road-vehicle localization with dual 2D push-broom lidars and 3D priors. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012. 2490–2497

  31. 31

    Pfrunder A, Borges P V K, Romero A R, et al. Real-time autonomous ground vehicle navigation in heterogeneous environments using a 3D lidar. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017. 2601–2608

  32. 32

    Liu Z Y, Yu S Y, Wang X, et al. Detecting drivable area for self-driving cars: an unsupervised approach. 2017. ArXiv: 1705.00451

  33. 33

    Satzoda R K, Sathyanarayana S, Srikanthan T, et al. Hierarchical additive hough transform for lane detection. IEEE Embedded Syst Lett, 2010, 2: 23–26

    Article  Google Scholar 

  34. 34

    Huang Y H, Chen S T, Chen Y, et al. Spatial-temproal based lane detection using deep learning. In: Proceedings of IFIP International Conference on Artificial Intelligence Applications and Innovations, 2018. 143–154

  35. 35

    Lee S, Kweon I S, Kim J, et al. Vpgnet: vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV), 2017. 1965–1973

  36. 36

    Pan X G, Shi J P, Luo P, et al. Spatial as deep: spatial CNN for traffic scene understanding. 2017. ArXiv: 1712.06080

  37. 37

    Zhang G, Zheng N N, Cui C, et al. An efficient road detection method in noisy urban environment. In: Proceedings of 2009 IEEE Intelligent Vehicles Symposium. New York: IEEE, 2009. 556–561

    Google Scholar 

  38. 38

    Lv X, Liu Z Y, Xin J M, et al. A novel approach for detecting road based on two-stream fusion fully convolutional network. In: Intelligent Vehicles. New York: IEEE, 2018

    Google Scholar 

  39. 39

    Chen Z, Chen Z J. Rbnet: a deep neural network for unified road and road boundary detection. In: Proceedings of International Conference on Neural Information Processing. Berlin: Springer, 2017. 677–687

    Google Scholar 

  40. 40

    Munoz-Bulnes J, Fernandez C, Parra I, et al. Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 366–371

  41. 41

    Lv X, Liu Z Y, Xin J M, et al. A novel approach for detecting road based on two-stream fusion fully convolutional network. In: Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV). New York: IEEE, 2018. 1464–1469

    Google Scholar 

  42. 42

    Warren C W. Fast path planning using modified A* method. In: Proceedings of IEEE International Conference on Robotics and Automation, 1993. 662–667

  43. 43

    Zeng W, Church R L. Finding shortest paths on real road networks: the case for A*. Int J Geographical Inf Sci, 2009, 23: 531–543

    Article  Google Scholar 

  44. 44

    Sislak D, Volf P, Pěchouček M. Accelerated A* path planning. In: Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, Budapest, 2009. 1133–1134

  45. 45

    LaValle S M. Rapidly-Exploring Random Trees: a New Tool for Path Planning. Technical Report (TR 98-11), Iowa State University, 1998

  46. 46

    Kuffner J J, LaValle S M. Rrt-connect: an efficient approach to single-query path planning. In: Proceedings of IEEE International Conference on Robotics and Automation, 2000. 995–1001

  47. 47

    Bohlin R, Kavraki L E. Path planning using lazy prm. In: Proceedings of IEEE International Conference on Robotics and Automation, 2000. 521–528

  48. 48

    Barraquand J, Langlois B, Latombe J C. Numerical potential field techniques for robot path planning. IEEE Trans Syst Man Cybern, 1992, 22: 224–241

    MathSciNet  Article  Google Scholar 

  49. 49

    Yang S X, Luo C. A neural network approach to complete coverage path planning. IEEE Trans Syst Man Cybern B, 2004, 34: 718–724

    Article  Google Scholar 

  50. 50

    Ferrer G, Sanfeliu A. Bayesian human motion intentionality prediction in urban environments. Pattern Recogn Lett, 2014, 44: 134–140

    Article  Google Scholar 

  51. 51

    Ghori O, Mackowiak R, Bautista M, et al. Learning to forecast pedestrian intention from pose dynamics. In: Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV), 2018

  52. 52

    Ma W-C, Huang D-A, Lee N, et al. Forecasting interactive dynamics of pedestrians with fictitious play. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 4636–4644

  53. 53

    Pfeiffer M, Schaeuble M, Nieto J, et al. From perception to decision: a data-driven approach to end-to-end motion planning for autonomous ground robots. In: Proceedings of 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017. 1527–1533

  54. 54

    Kim B, Kang C M, Lee S H, et al. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network. 2017. ArXiv: 1704.07049

  55. 55

    Takahashi A, Hongo T, Ninomiya Y, et al. Local path planning and motion control for agv in positioning. In: Proceedings of IEEE/RSJ International Workshop on Intelligent Robots and Systems, 1989. 392–397

  56. 56

    Piazzi A, Bianco C G L. Quintic g/sup 2/-splines for trajectory planning of autonomous vehicles. In: Proceedings of the IEEE Intelligent Vehicles Symposium, 2000. 198–203

  57. 57

    Komoriya K, Tanie K. Trajectory design and control of a wheel-type mobile robot using b-spline curve. In: Proceedings of IEEE/RSJ International Workshop on Intelligent Robots and Systems, 1989. 398–405

  58. 58

    Holger B, Dennis N, Marius Z J, et al. From G2 to G3 continuity: continuous curvature rate steering functions for sampling-based nonholonomic motion planning. In: Proceedings of Intelligent Vehicles. New York: IEEE, 2018

    Google Scholar 

  59. 59

    Petereit J, Emter T, Frey C W, et al. Application of hybrid A* to an autonomous mobile robot for path planning in unstructured outdoor environments. In: Proceedings of the 7th German Conference on Robotics, 2012. 1–6

  60. 60

    Veres S M, Molnar L, Lincoln N K, et al. Autonomous vehicle control systemsa review of decision making. In: Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 2011. 225: 155–195

  61. 61

    Lee D, Yannakakis M. Principles and methods of testing finite state machines-a survey. Proc IEEE, 1996, 84: 1090–1123

    Article  Google Scholar 

  62. 62

    Montemerlo M, Becker J, Bhat S, et al. Junior: the stanford entry in the urban challenge. J Field Robot, 2008, 25: 569–597

    Article  Google Scholar 

  63. 63

    Feinberg E A, Shwartz A. Handbook of Markov Decision Processes: Methods and Applications. Berlin: Springer Science & Business Media, 2012

    Google Scholar 

  64. 64

    Ulbrich S, Maurer M. Probabilistic online pomdp decision making for lane changes in fully automated driving. In: Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems (ITSC), 2013. 2063–2067

  65. 65

    Brechtel S, Gindele T, Dillmann R. Probabilistic decision-making under uncertainty for autonomous driving using continuous pomdps. In: Proceedings of IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), 2014. 392–399

  66. 66

    van Otterlo M, Wiering M. Reinforcement learning and markov decision processes. In: Proceedings of Reinforcement Learning. Belin: Springer, 2012. 3–42

    Google Scholar 

  67. 67

    Morton J, Wheeler T A, Kochenderfer M J. Analysis of recurrent neural networks for probabilistic modeling of driver behavior. IEEE Trans Intell Transp Syst, 2017, 18: 1289–1298

    Article  Google Scholar 

  68. 68

    Xu L H, Wang Y Z, Sun H B, et al. Integrated longitudinal and lateral control for Kuafu-II autonomous vehicle. IEEE Trans Intell Transp Syst, 2016, 17: 2032–2041

    Article  Google Scholar 

  69. 69

    Coulter R C. Implementation of the Pure Pursuit Path Tracking Algorithm. Technical Report, Carnegie-Mellon UNIV Pittsburgh PA Robotics INST, 1992

  70. 70

    Camacho E F, Alba C B. Model Predictive Control. Berlin: Springer Science & Business Media, 2013

  71. 71

    Rasekhipour Y, Khajepour A, Chen S K, et al. A potential field-based model predictive path-planning controller for autonomous road vehicles. IEEE Trans Intell Transp Syst, 2017, 18: 1255–1267

    Article  Google Scholar 

  72. 72

    Varshney P K. Multisensor data fusion. Electron Commun Eng J, 1997, 9: 245–253

    Article  Google Scholar 

  73. 73

    Hall D L, Llinas J. An introduction to multisensor data fusion. Proc IEEE, 1997, 85: 6–23

    Article  Google Scholar 

  74. 74

    Zhang Q, Liu Y, Blum R S, et al. Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: a review. Inf Fusion, 2018, 40: 57–75

    Article  Google Scholar 

  75. 75

    Liu Y H, Fan X Q, Lv C, et al. An innovative information fusion method with adaptive Kalman filter for integrated INS/GPS navigation of autonomous vehicles. Mech Syst Signal Process, 2018, 100: 605–616

    Article  Google Scholar 

  76. 76

    Behrendt K, Novak L, Botros R. A deep learning approach to traffic lights: detection, tracking, and classification. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2017. 1370–1377

  77. 77

    Haberjahn M, Kozempel K. Multi level fusion of competitive sensors for automotive environment perception. In: Proceedings of 16th International Conference on Information Fusion (FUSION), 2013. 397–403

  78. 78

    Scheunert U, Lindner P, Richter E, et al. Early and multi level fusion for reliable automotive safety systems. In: Proceedings of Intelligent Vehicles Symposium. New York: IEEE, 2007. 196–201

    Google Scholar 

  79. 79

    Rodrfguez-Garavito C H, Ponz A, García F, et al. Automatic laser and camera extrinsic calibration for data fusion using road plane. In: Proceedings of the 17th International Conference on Information Fusion (FUSION), 2014. 1–6

  80. 80

    Park Y, Yun S, Won C S, et al. Calibration between color camera and 3D LIDAR instruments with a polygonal planar board. Sensors, 2014, 14: 5333–5353

    Article  Google Scholar 

  81. 81

    Wang X, Xu L H, Sun H B, et al. On-road vehicle detection and tracking using MMW radar and monovision fusion. IEEE Trans Intell Transp Syst, 2016, 17: 2075–2084

    Article  Google Scholar 

  82. 82

    Wang T, Xin J M, Zheng N N. A method integrating human visual attention and consciousness of radar and vision fusion for autonomous vehicle navigation. In: Proceedings of IEEE 4th International Conference on Space Mission Challenges for Information Technology (SMC-IT), 2011. 192–197

  83. 83

    Zhu Z, Liu J L. Unsupervised extrinsic parameters calibration for multi-beam lidars. In: Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering, Paris, 2013. 1110–1113

  84. 84

    Jiang J J, Xue P X, Chen S T, et al. Line feature based extrinsic calibration of lidar and camera. In: Proceedings of 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), 2018. 1–6

  85. 85

    Sun S L, Deng Z L. Multi-sensor optimal information fusion Kalman filter. Automatica, 2004, 40: 1017–1023

    MathSciNet  Article  MATH  Google Scholar 

  86. 86

    Sarkka S, Vehtari A, Lampinen J. Rao-blackwellized particle filter for multiple target tracking. Inf Fusion, 2007, 8: 2–15

    Article  Google Scholar 

  87. 87

    Yang G S, Lin Y, Bhattacharya P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Inf Sci, 2010, 180: 1942–1954

    Article  Google Scholar 

  88. 88

    Li Y B, Chen J, Ye F, et al. The improvement of DS evidence theory and its application in IR/MMW target recognition. J Sens, 2016, 2016: 1–15

    Google Scholar 

  89. 89

    Wu H D, Siegel M, Stiefelhagen R, et al. Sensor fusion using dempster-shafer theory [for context-aware hci]. In: Proceedings of the 19th IEEE Instrumentation and Measurement Technology Conference, 2002. 7–12

  90. 90

    Murphy R R. Dempster-Shafer theory for sensor fusion in autonomous mobile robots. IEEE Trans Robot Automat, 1998, 14: 197–206

    Article  Google Scholar 

  91. 91

    Subramanian V, Burks T F, Dixon W E. Sensor fusion using fuzzy logic enhanced Kalman filter for autonomous vehicle guidance in citrus groves. Trans ASABE, 2009, 52: 1411–1422

    Article  Google Scholar 

  92. 92

    Klein L A, Klein L A. Sensor and data fusion: a tool for information assessment and decision making. In: Proceedings of SPIE, 2004

  93. 93

    Eslami S M A, Rezende D J, Besse F, et al. Neural scene representation and rendering. Science, 2018, 360: 1204–1210

    Article  Google Scholar 

  94. 94

    Chen S T, Shang J H, Zhang S Y, et al. Cognitive map-based model: toward a developmental framework for self-driving cars. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 1–8

  95. 95

    Chen S T, Zhang S Y, Shang J H, et al. Brain-inspired cognitive model with attention for self-driving cars. IEEE Trans Cogn Dev Syst, 2019, 11: 13–25

    Article  Google Scholar 

  96. 96

    Li D Y, Gao H B. A hardware platform framework for an intelligent vehicle based on a driving brain. Engineering, 2018, 4: 464–470

    Article  Google Scholar 

  97. 97

    Chen L. The topological approach to perceptual organization. Visual Cognition, 2005, 12: 553–637

    Article  Google Scholar 

  98. 98

    Eslami S M A, Heess N, Weber T, et al. Attend, infer, repeat: fast scene understanding with generative models. In: Proceedings of the 30th Conference on Neural Information Processing Systems, Barcelona, 2016. 3225–3233

  99. 99

    Bar-Shalom Y, Daum F, Huang J. The probabilistic data association filter. IEEE Control Syst, 2009, 29: 82–100

    Article  MATH  Google Scholar 

  100. 100

    Svensson L, Svensson D, Guerriero M, et al. Set JPDA filter for multitarget tracking. IEEE Trans Signal Process, 2011, 59: 4677–4691

    MathSciNet  Article  MATH  Google Scholar 

  101. 101

    Blackman S S. Multiple hypothesis tracking for multiple target tracking. IEEE Aerosp Electron Syst Mag, 2004, 19: 5–18

    Article  Google Scholar 

  102. 102

    Kim C, Li F X, Ciptadi A, et al. Multiple hypothesis tracking revisited. In: Proceedings of the IEEE International Conference on Computer Vision, 2015. 4696–4704

  103. 103

    Kuhn H W. The Hungarian method for the assignment problem. Naval Res Logistics, 1955, 2: 83–97

    MathSciNet  Article  MATH  Google Scholar 

  104. 104

    Cho H, Seo Y-W, Kumar B V K V, et al. A multi-sensor fusion system for moving object detection and tracking in urban driving environments. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2014. 1836–1843

  105. 105

    Chavez-Garcia R O, Aycard O. Multiple sensor fusion and classification for moving object detection and tracking. IEEE Trans Intell Transp Syst, 2016, 17: 525–534

    Article  Google Scholar 

  106. 106

    Göhring D, Wang M, Schnürmacher M, et al. Radar/lidar sensor fusion for car-following on highways. In: Proceedings of the 5th International Conference on Automation, Robotics and Applications (ICARA), 2011. 407–412

  107. 107

    Fayad F, Cherfaoui V. Object-level fusion and confidence management in a multi-sensor pedestrian tracking system. In: Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2008. 58–63

  108. 108

    Kim D Y, Jeon M. Data fusion of radar and image measurements for multi-object tracking via Kalman filtering. Inf Sci, 2014, 278: 641–652

    MathSciNet  Article  Google Scholar 

  109. 109

    Govaers F, Koch W. An exact solution to track-to-track-fusion at arbitrary communication rates. IEEE Trans Aerosp Electron Syst, 2012, 48: 2718–2729

    Article  Google Scholar 

  110. 110

    Zhang Z Y, Fidler S, Urtasun R. Instance-level segmentation for autonomous driving with deep densely connected mrfs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 669–677

  111. 111

    Sun Y X, Liu M, Meng M Q H. Improving RGB-D SLAM in dynamic environments: a motion removal approach. Robot Auton Syst, 2017, 89: 110–122

    Article  Google Scholar 

  112. 112

    Sun Y X, Liu M, Meng M Q H. Motion removal for reliable RGB-D SLAM in dynamic environments. Robotics Autonomous Syst, 2018, 108: 115–128

    Article  Google Scholar 

  113. 113

    Caron F, Duflos E, Pomorski D, et al. GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects. Inf Fusion, 2006, 7: 221–230

    Article  Google Scholar 

  114. 114

    Suhr J K, Jang J, Min D, et al. Sensor fusion-based low-cost vehicle localization system for complex urban environments. IEEE Trans Intell Transp Syst, 2017, 18: 1078–1086

    Article  Google Scholar 

  115. 115

    Wan G W, Yang X L, Cai R L, et al. Robust and precise vehicle localization based on multi-sensor fusion in diverse city scenes. 2017. ArXiv: 1711.05805

  116. 116

    Tamar A, Wu Y, Thomas G, et al. Value iteration networks. In: Advances in Neural Information Processing Systems, 2016. 2154–2162

  117. 117

    Katsuki F, Constantinidis C. Bottom-up and top-down attention: different processes and overlapping neural systems. Neuroscientist, 2014, 20: 509–521

    Article  Google Scholar 

  118. 118

    Miller E K. Neurobiology: straight from the top. Nature, 1999, 401: 650–651

    Article  Google Scholar 

  119. 119

    Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Machine Intell, 1998, 20: 1254–1259

    Article  Google Scholar 

  120. 120

    Kadir T, Brady M. Saliency, scale and image description. Int J Comput Vision, 2001, 45: 83–105

    Article  MATH  Google Scholar 

  121. 121

    Ba J, Mnih V, Kavukcuoglu K. Multiple object recognition with visual attention. 2014. ArXiv: 1412.7755

  122. 122

    Hu J, Shen L, Sun G. Squeeze-and-excitation networks. 2017 ArXiv: 1709.01507

  123. 123

    Fu J L, Zheng H L, Mei T. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. 3

  124. 124

    Kuo W J, Sjostrom T, Chen Y P, et al. Intuition and deliberation: two systems for strategizing in the brain. Science, 2009, 324: 519–522

    Article  Google Scholar 

  125. 125

    Zheng N N, Liu Z Y, Ren P J, et al. Hybrid-augmented intelligence: collaboration and cognition. Front Inf Technol Electron Eng, 2017, 18: 153–179

    Article  Google Scholar 

  126. 126

    Zhao D B, Hu Z H, Xia Z P, et al. Full-range adaptive cruise control based on supervised adaptive dynamic programming. Neurocomputing, 2014, 125: 57–67

    Article  Google Scholar 

  127. 127

    Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518: 529–533

    Article  Google Scholar 

  128. 128

    Silver D, Huang A, Maddison C J, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, 529: 484–489

    Article  Google Scholar 

  129. 129

    Gupta S, Davidson J, Levine S, et al. Cognitive mapping and planning for visual navigation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 7272–7281

  130. 130

    Rusu A A, Rabinowitz N C, Desjardins G, et al. Progressive neural networks. 2016. ArXiv: 1606.04671

  131. 131

    Rusu A A, Vecerik M, Rothorl T, et al. Sim-to-real robot learning from pixels with progressive nets. 2016. ArXiv: 1610.04286

Download references

Acknowledgements

This work was partially supported by National Natural Science Foundation of China (Grant Nos. 61773312, 61790563).

Author information

Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049, China

    Shitao Chen, Zhiqiang Jian, Yuhao Huang, Yu Chen, Zhuoli Zhou & Nanning Zheng

  2. National Engineering Laboratory for Visual Information Processing and Applications, Xi’an, 710049, China

    Shitao Chen, Zhiqiang Jian, Yuhao Huang, Yu Chen, Zhuoli Zhou & Nanning Zheng

Authors
  1. Shitao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiqiang Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuhao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhuoli Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nanning Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nanning Zheng.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Jian, Z., Huang, Y. et al. Autonomous driving: cognitive construction and situation understanding. Sci. China Inf. Sci. 62, 81101 (2019). https://doi.org/10.1007/s11432-018-9850-9

Download citation

Keywords

  • autonomous driving
  • event-driven mechanism
  • cognitive construction
  • situation understanding
  • intuitive reasoning