Deep Learning for Driverless Vehicles

Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 136)


Automation is becoming a large component of many industries in the 21st century, in areas ranging from manufacturing, communications and transportation. Automation has offered promised returns of improvements in safety, productivity and reduced costs. Many industry leaders are specifically working on the application of autonomous technology in transportation to produce “driverless” or fully autonomous vehicles. A key technology that has the potential to drive the future development of these vehicles is deep learning. Deep learning has been an area of interest in machine learning for decades now but has only come into widespread application in recent years. While traditional analytical control systems and computer vision techniques have in the past been adequate for the fundamental proof of concept of autonomous vehicles, this review of current and emerging technologies demonstrates these short comings and the road map for overcoming them with deep learning.


Deep learning Autonomous Driverless Convolutional Neural networks Machine learning Machine vision 


  1. 1.
    World Health Organisation, Road traffic deaths—2013. Accessed 26 Mar 2017
  2. 2.
    World Health Organisation, World Report on Road Traffic Injury Prevention (World Health Organisation, Geneva, Switzerland, 2004)Google Scholar
  3. 3.
    Mine Safety and Health Administration, Mining Industry Accident, Injuries, Employment, and Production Statistics and Reports, 3 Mar 2017.
  4. 4.
    Fortescue Metals Group Limited, Investor Briefing 2016, 25 Oct 2016. Accessed 15 Nov 2016
  5. 5.
    A. Topf, Canadian oil sands giant testing autonomous haul trucks (2016). Accessed 5 May 2017
  6. 6.
    Business Wire, Peloton Announces U.S. Department of Energy-funded Team to Cut Heavy Truck Fuel Use by 20 Percent with Smart Powertrains and Platooning, 14 Nov 2016.
  7. 7.
    U.S Airforce, Global Positioning System Standard Positioning Service Performance Standard (2008). Accessed 20 May 2017
  8. 8.
    Applanix, Position and Orientation System for Land Vehicles—Datasheet, 26 Mar 2017.
  9. 9.
    S. Ericson, Vision-Based Perception for Localisation of Autonomous Agricultural Robots (2017)Google Scholar
  10. 10.
    K. Jo, K. Chu, K. Lee, M. Sunwoo, Integration of multiple vehicle models with an IMM filter for vehicle localization, San Diego (2010)Google Scholar
  11. 11.
    J.A. Reeds, L.A. Shepp, Optimal paths for a car that goes both forwards and backwards. Pac. J. Math 145(2), 367–393 (1990)MathSciNetCrossRefGoogle Scholar
  12. 12.
    D. Ferguson, A. Stentz, The field D* algorithm for improved path planning and replanning in uniform and non-uniform cost environments (2005)Google Scholar
  13. 13.
    P.E. Hart, N.J. Nilsson, B. Raphael, A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybern. 4(2), 100–107 (1968)CrossRefGoogle Scholar
  14. 14.
    K. Chu, M. Lee, M. Sunwoo, Local path planning for off-road autonomous driving with avoidance of static obstacles. Intell. Trans. Syst. 13(4), 1599–1616 (2012)CrossRefGoogle Scholar
  15. 15.
    D. Dmitri, S. Thrun, M. Montemerlo, J. Diebel, Path planning for autonomous vehicles in unknown semi-structured environments. Int. J. Robot. Res. 29(5), 485–501 (2010)CrossRefGoogle Scholar
  16. 16.
    J. Roberts, P. Corke, Obstacle detection for a mining vehicle using a 2D laser, in Proceedings of the Australian Conference on Robotics (2000)Google Scholar
  17. 17.
    A. Stentz, A. Kelly, P. Rander, H. Herman, O. Amidi, Real-time, multi-perspective perception for unmanned ground vehicles (2003)Google Scholar
  18. 18.
    J. Han, D. Kim, M. Lee, M. Sunwoo, Enhanced road boundary and obstacle detection using a downward-looking LIDAR sensor. Trans. Veh. Technol. 61(3), 971–985 (2012)CrossRefGoogle Scholar
  19. 19.
    J. Tang, A. Zakhor, 3D object detection from roadside data using laser scanners, in Three-Dimensional Imaging, Interaction, and Measurement (2011)Google Scholar
  20. 20.
    J.R.R. Uijings, K.A. van de Sande, T. Gevers, A.W.M. Smeulders, Selective search for object recognition. Int. J. Comput. Vis. 104(2), 154–171 (2013)CrossRefGoogle Scholar
  21. 21.
    Y. Huang, S. Liu, Multi-class obstacle detection and classification using stereovision and improved active contour models. Inst. Eng. Technol. 10(3), 197–205 (2016)Google Scholar
  22. 22.
    U. Ozguner, C. Stiller, K. Redmill, Systems for safety and autonomous behavior in cars: the DARPA grand challenge experience. Proc. IEEE 95(2), 397–412 (2007)CrossRefGoogle Scholar
  23. 23.
    L.D. Jackel, D. Sharman, C.E. Stenard, I.B. Strom, D. Zucket, Optical character recognition for self-service banking (1995)Google Scholar
  24. 24.
    D. Pomerleau, T. Jochem, Rapidly adapting machine vision for automated vehicle steering. IEEE Expert 19–27 (1996)CrossRefGoogle Scholar
  25. 25.
    E. Ackerman, How is mastering autonomous driving with deep learning, 10 Mar 2017. Accessed 26 Apr 2017
  26. 26.
    Mobileye, The Three Pillars of Autonomous Driving, 20 June 2016. Accessed 20 Apr 2017
  27. 27.
    X. Wang, L. Gao, S. Mao, S. Pandey, CSI-based fingerprinting for indoor localisation: a deep learning approach. IEEE Trans. Veh. Technol. 66(1) (2016)Google Scholar
  28. 28.
    W. Zhang, K. Liu, W. Zhang, Y. Zhang, J. Gu, Deep neural networks for wireless localization in indoor and outdoor environments. Neurocomputing 194, 279–287 (2016)CrossRefGoogle Scholar
  29. 29.
    K.-W. Chen, C.-H. Wang, X. Wei, Q. Liang, C.-S. Chen, Vision-based positioning for internet-of-vehicles. IEEE Trans. Intell. Transp. Syst. 18(2), 364–376 (2017)CrossRefGoogle Scholar
  30. 30.
    R. Hadsell, A. Erkan, P. Sermanet, M. Scoffier, U. Muller, Y. LeCun, Deep belief net learning in a long-range vision system for autonomous off-road driving, in International Conference on Intelligent Robotic Systems (2008), pp. 628–633Google Scholar
  31. 31.
    A. Krizhevsky, I. Sutskever, G. Hinton, ImageNet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 2, 1097–1105 (2012)Google Scholar
  32. 32.
    Stanford Vision Lab, ImageNet (2017)Google Scholar
  33. 33.
    P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun, OverFeat: integrated recognition, localization and detection using convolutional networks (2013)Google Scholar
  34. 34.
    R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in Computer Vision and Pattern Recognition (2014)Google Scholar
  35. 35.
    K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, in International Conference on Learning Representations (2015)Google Scholar
  36. 36.
    B. Huval, T. Wang, S. Tandon, J. Kiske, W. Song, J. Pazhayampallil, M. Andriluka, P. Rajpurkar, T. Migimatsu, R. Cheng-Yue, F. Mujica, A. Coates, A. Ng, An empirical evaluation of deep learning on highway driving (2015)Google Scholar
  37. 37.
    R. Qian, B. Zhang, Y. Yue, Z. Wang, F. Coenen, Robust Chinese traffic sign detection and recognition with deep convolutional neural network, in International Conference on Natural Computation (2015), pp. 791–796Google Scholar
  38. 38.
    D. Ciresan, U. Meier, J. Masci, J. Schmidhuber, A committee of neural networks for traffic sign classification, in Proceedings of International Joint Conference on Neural Networks (2011), pp. 1918–1921Google Scholar
  39. 39.
    R. Girshick, Fast R-CNN, in International Conference on Computer Vision (2015)Google Scholar
  40. 40.
    Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li, S. Hu, Traffic-sign detection and classification in the wild, in Computer Vision and Pattern Recognition (2016)Google Scholar
  41. 41.
    D.V. Prokhorov, Road obstacle classification with attention windows, in Intelligent Vehicles Symposium, June 2010Google Scholar
  42. 42.
    M. Teichmann, M. Weber, M. Zollner, R. Cipolla, R. Urtasun, Real-time joint semantic reasoning for autonomous driving (2016)Google Scholar
  43. 43.
    A. Geiger, P. Lenz, R. Urtasun, Are we ready for autonomous driving? The KITTI vision benchmark suite, in IEEE CVPR (2012)Google Scholar
  44. 44.
    J.-G. Wang, L. Zhou, Y. Pan, S. Lee, Z. Song, B.S. Han, V.B. Saputra, Appearance-based Brake-Lights recognition using deep learning and vehicle detection, in IEEE Intelligent Vehicle Symposium (IV), 19–22 June 2016Google Scholar
  45. 45.
    Nvidia Corporation, CES 2016: NVIDIA DRIVENet Demo—Visualizing a Self-Driving Future (Part 5), 1 Jan 2016. Accessed 9 Apr 2017
  46. 46.
    Mobileye, Autonomous Car—What Goes into Sensing for Autonomous Driving? (2016). Accessed 28 Apr 2017
  47. 47.
    Net-Scale Technologies Inc, Autonomous Off-Road Vehicle Control Using End-to-End Learning (DARPA/CMO, Arlington, VA, 2004)Google Scholar
  48. 48.
    M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, End to end learning for self-driving cars (2016)Google Scholar
  49. 49.
    A. El Sallab, M. Abdou, E. Perot, S. Yogamani, End-to-end deep reinforcement learning for lane keeping assist, Presented at the Machine Learning for Intelligent Transportation Systems Workshop (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer Science and Software EngineeringUniversity of Western AustraliaCrawleyAustralia
  2. 2.School of Electrical Engineering, Computing and Mathematical SciencesCurtin UniversityBentleyAustralia
  3. 3.School of Computing and CommunicationsLancaster UniversityBailriggUK

Personalised recommendations