Safe Deep Neural Network-Driven Autonomous Vehicles Using Software Safety Cages

  • Sampo KuuttiEmail author
  • Richard Bowden
  • Harita Joshi
  • Robert de Temple
  • Saber Fallah
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11872)


Deep learning is a promising class of techniques for controlling an autonomous vehicle. However, functional safety validation is seen as a critical issue for these systems due to the lack of transparency in deep neural networks and the safety-critical nature of autonomous vehicles. The black box nature of deep neural networks limits the effectiveness of traditional verification and validation methods. In this paper, we propose two software safety cages, which aim to limit the control action of the neural network to a safe operational envelope. The safety cages impose limits on the control action during critical scenarios, which if breached, change the control action to a more conservative value. This has the benefit that the behaviour of the safety cages is interpretable, and therefore traditional functional safety validation techniques can be applied. The work here presents a deep neural network trained for longitudinal vehicle control, with safety cages designed to prevent forward collisions. Simulated testing in critical scenarios shows the effectiveness of the safety cages in preventing forward collisions whilst under normal highway driving unnecessary interruptions are eliminated, and the deep learning control policy is able to perform unhindered. Interventions by the safety cages are also used to re-train the network, resulting in a more robust control policy.


Automatic control Autonomous vehicles Cyber-physical systems Deep learning Safety 



This work was supported by the UK-EPSRC grant EP/R512217/1 and Jaguar Land Rover.


  1. 1.
    Adler, R., Feth, P., Schneider, D.: Safety engineering for autonomous vehicles. In: 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshop (DSN-W), pp. 200–205. IEEE (2016)Google Scholar
  2. 2.
    Archer, J.: Indicators for traffic safety assessment and prediction and their application in micro-simulation modelling: a study of urban and suburban intersections. Ph.D. thesis, KTH (2005)Google Scholar
  3. 3.
    Burton, S., Gauerhof, L., Heinzemann, C.: Making the case for safety of machine learning in highly automated driving. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2017. LNCS, vol. 10489, pp. 5–16. Springer, Cham (2017). Scholar
  4. 4.
    Department for Transport: Research on the Impacts of Connected and Autonomous Vehicles (CAVs) on Traffic Flow: Summary Report (2017)Google Scholar
  5. 5.
    Glaser, S., Vanholme, B., Mammar, S., Gruyer, D., Nouveliere, L.: Maneuver-based trajectory planning for highly autonomous vehicles on real road with traffic and driver interaction. IEEE Trans. Intell. Transp. Syst. 11(3), 589–606 (2010)CrossRefGoogle Scholar
  6. 6.
    Haddadin, S., et al.: On making robots understand safety: embedding injury knowledge into control. Int. J. Robot. Res. 31(13), 1578–1602 (2012)CrossRefGoogle Scholar
  7. 7.
    Heckemann, K., Gesell, M., Pfister, T., Berns, K., Schneider, K., Trapp, M.: Safe automotive software. In: König, A., Dengel, A., Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds.) KES 2011. LNCS (LNAI), vol. 6884, pp. 167–176. Springer, Heidelberg (2011). Scholar
  8. 8.
    International Organization for Standardization: ISO 26262: Road vehicles-functional safety. International Standard ISO/FDIS (2011)Google Scholar
  9. 9.
    IPG Automotive GmbH: Carmaker: Virtual testing of automobiles and light-duty vehicles (2017).
  10. 10.
    Kalra, N., Paddock, S.M.: Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? Transp. Res. Part A Policy Pract. 94, 182–193 (2016)CrossRefGoogle Scholar
  11. 11.
    Koopman, P., Wagner, M.: Challenges in autonomous vehicle testing and validation. SAE Int. J. Transp. Saf. 4(1), 15–24 (2016)CrossRefGoogle Scholar
  12. 12.
    Kuffner Jr., J.J., Anderson-Sprecher, P.E.: Virtual safety cages for robotic devices, US Patent 9,522,471, 20 December 2016Google Scholar
  13. 13.
    Kuutti, S., Fallah, S., Bowden, R., Barber, P.: Deep Learning for Autonomous Vehicle Control: Algorithms, State-of-the-Art, and Future Prospects. Morgan & Claypool Publishers, San Rafael (2019)Google Scholar
  14. 14.
    Kuutti, S., Fallah, S., Katsaros, K., Dianati, M., Mccullough, F., Mouzakitis, A.: A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications. IEEE Internet Things J. 5(2), 829–846 (2018)CrossRefGoogle Scholar
  15. 15.
    Lu, J., Dissanayake, S., Xu, L., Williams, K.: Safety evaluation of right-turns followed by u-turns as an alternative to direct left turns: Crash data analysis. Florida Department of Transportation (2001)Google Scholar
  16. 16.
    Montanaro, U., et al.: Towards connected autonomous driving: review of use-cases. Veh. Syst. Dyn. 57, 1–36 (2018)Google Scholar
  17. 17.
    Polycarpou, M., Zhang, X., Xu, R., Yang, Y., Kwan, C.: A neural network based approach to adaptive fault tolerant flight control. In: Proceedings of the 2004 IEEE International Symposium on Intelligent Control, pp. 61–66. IEEE (2004)Google Scholar
  18. 18.
    Ross, S., Gordon, G., Bagnell, D.: A reduction of imitation learning and structured prediction to no-regret online learning. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 627–635 (2011)Google Scholar
  19. 19.
    Thrun, S.: Toward robotic cars. Commun. ACM 53(4), 99 (2010)CrossRefGoogle Scholar
  20. 20.
    Treiber, M., Kesting, A.: Car-following models based on driving strategies. In: Traffic Flow Dynamics, pp. 181–204. Springer, Heidelberg (2013). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Connected Autonomous Vehicles LabUniversity of SurreyGuildfordUK
  2. 2.Centre for Vision, Speech and Signal ProcessingUniversity of SurreyGuildfordUK
  3. 3.Jaguar Land RoverCoventryUK

Personalised recommendations