Skip to main content

Lab Conditions for Research on Explainable Automated Decisions

  • Conference paper
  • First Online:
Trustworthy AI - Integrating Learning, Optimization and Reasoning (TAILOR 2020)

Abstract

Artificial neural networks are being proposed for automated decision making under uncertainty in many visionary contexts, including high-stake tasks such as navigating autonomous cars through dense traffic. Against this background, it is imperative that the decision making entities meet central societal desiderata regarding dependability, perspicuity, explainability, and robustness. Decision making problems under uncertainty are typically captured formally as variations of Markov decision processes (MDPs). This paper discusses a set of natural and easy-to-control abstractions, based on the Racetrack benchmarks and extensions thereof, that altogether connect the autonomous driving challenge to the modelling world of MDPs. This is then used to study the dependability and robustness of NN-based decision entities, which in turn are based on state-of-the-art NN learning techniques. We argue that this approach can be regarded as providing laboratory conditions for a systematic, structured and extensible comparative analysis of NN behavior, of NN learning performance, as well as of NN verification and analysis techniques.

Authors are listed alphabetically. This work was partially supported by the German Research Foundation (DFG) under grant No. 389792660, as part of TRR 248, see https://perspicuous-computing.science, by the ERC Advanced Investigators Grant 695614 (POWVER), and by the Key-Area Research and Development Program Grant 2018B010107004 of Guangdong Province.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Baier, C., Dubslaff, C., Hermanns, H., Klauck, M., Klüppelholz, S., Köhl, M.A.: Components in probabilistic systems: suitable by construction. In: Margaria, T., Steffen, B. (eds.) ISoLA 2020. LNCS, vol. 12476, pp. 240–261. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61362-4_13

    Chapter  Google Scholar 

  2. Barnaghi, P., Ganz, F., Henson, C., Sheth, A.: Computing perception from sensor data. In: SENSORS, 2012 IEEE, pp. 1–4. IEEE (2012)

    Google Scholar 

  3. Barto, A.G., Bradtke, S.J., Singh, S.P.: Learning to act using real-time dynamic programming. Artif. Intell. 72(1–2), 81–138 (1995). https://doi.org/10.1016/0004-3702(94)00011-O

    Article  Google Scholar 

  4. Berndt, H., Emmert, J., Dietmayer, K.: Continuous driver intention recognition with hidden Markov models. In: 11th International IEEE Conference on Intelligent Transportation Systems, ITSC 2008, Beijing, China, 12–15 October 2008, pp. 1189–1194. IEEE (2008). https://doi.org/10.1109/ITSC.2008.4732630

  5. Bonet, B., Geffner, H.: Labeled RTDP: improving the convergence of real-time dynamic programming. In: Giunchiglia, E., Muscettola, N., Nau, D.S. (eds.) Proceedings of the Thirteenth International Conference on Automated Planning and Scheduling (ICAPS 2003), Trento, Italy, 9–13 June 2003, pp. 12–21. AAAI (2003). http://www.aaai.org/Library/ICAPS/2003/icaps03-002.php

  6. Budde, C.E., D’Argenio, P.R., Hartmanns, A., Sedwards, S.: A statistical model checker for nondeterminism and rare events. In: Beyer, D., Huisman, M. (eds.) TACAS 2018. LNCS, vol. 10806, pp. 340–358. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-89963-3_20

    Chapter  Google Scholar 

  7. Christakis, M., et al.: Automated Safety Verification of Programs Invoking Neural Networks (2020). Submitted for publication

    Google Scholar 

  8. Dietmayer, K.: Predicting of machine perception for automated driving. In: Maurer, M., Gerdes, J.C., Lenz, B., Winner, H. (eds.) Autonomous Driving, pp. 407–424. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-48847-8_20

    Chapter  Google Scholar 

  9. Faqeh, R., et al.: Towards dynamic dependable systems through evidence-based continuous certification. In: Margaria, T., Steffen, B. (eds.) ISoLA 2020. LNCS, vol. 12477, pp. 416–439. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61470-6_25

    Chapter  Google Scholar 

  10. Gardner, M.: Mathematical games. Sci. Am. 229, 118–121 (1973)

    Article  Google Scholar 

  11. Gros, T.P., Groß, D., Gumhold, S., Hoffmann, J., Klauck, M., Steinmetz, M.: TraceVis: towards visualization for deep statistical model checking. In: Proceedings of the 9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation. From Verification to Explanation (2020)

    Google Scholar 

  12. Gros, T.P., Hermanns, H., Hoffmann, J., Klauck, M., Steinmetz, M.: Deep statistical model checking. In: Gotsman, A., Sokolova, A. (eds.) FORTE 2020. LNCS, vol. 12136, pp. 96–114. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50086-3_6

    Chapter  Google Scholar 

  13. Gumhold, S.: The computer graphics and visualization framework. https://github.com/sgumhold/cgv. Accessed 18 May 2020

  14. McMahan, H.B., Gordon, G.J.: Fast exact planning in Markov decision processes. In: Biundo, S., Myers, K.L., Rajan, K. (eds.) Proceedings of the Fifteenth International Conference on Automated Planning and Scheduling (ICAPS 2005), 5–10 June 2005, Monterey, California, USA, pp. 151–160. AAAI (2005). http://www.aaai.org/Library/ICAPS/2005/icaps05-016.php

  15. Meresht, V.B., De, A., Singla, A., Gomez-Rodriguez, M.: Learning to switch between machines and humans. CoRR abs/2002.04258 (2020). https://arxiv.org/abs/2002.04258

  16. Pineda, L.E., Lu, Y., Zilberstein, S., Goldman, C.V.: Fault-tolerant planning under uncertainty. In: Rossi, F. (ed.) IJCAI 2013, Proceedings of the 23rd International Joint Conference on Artificial Intelligence, Beijing, China, 3–9 August 2013, pp. 2350–2356. IJCAI/AAAI (2013). http://www.aaai.org/ocs/index.php/IJCAI/IJCAI13/paper/view/6819

  17. Pineda, L.E., Zilberstein, S.: Planning under uncertainty using reduced models: revisiting determinization. In: Chien, S.A., Do, M.B., Fern, A., Ruml, W. (eds.) Proceedings of the Twenty-Fourth International Conference on Automated Planning and Scheduling, ICAPS 2014, Portsmouth, New Hampshire, USA, 21–26 June 2014. AAAI (2014). http://www.aaai.org/ocs/index.php/ICAPS/ICAPS14/paper/view/7920

  18. Puterman, M.L.: Markov decision processes: discrete stochastic dynamic programming. In: Wiley Series in Probability and Statistics. Wiley (1994). https://doi.org/10.1002/9780470316887

  19. Sadri, F.: Logic-based approaches to intention recognition. In: Handbook of Research on Ambient Intelligence and Smart Environments: Trends and Perspectives, pp. 346–375. IGI Global (2011)

    Google Scholar 

  20. Strickland, M., Fainekos, G.E., Amor, H.B.: Deep predictive models for collision risk assessment in autonomous driving. In: 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, 21–25 May 2018, pp. 1–8. IEEE (2018). https://doi.org/10.1109/ICRA.2018.8461160

  21. Tahboub, K.A.: Intelligent human-machine interaction based on dynamic Bayesian networks probabilistic intention recognition. J. Intell. Robotic Syst. 45(1), 31–52 (2006). https://doi.org/10.1007/s10846-005-9018-0

    Article  Google Scholar 

  22. Wissenschaftsrat: Perspektiven der Informatik in Deutschland, October 2020. https://www.wissenschaftsrat.de/download/2020/8675-20.pdf

  23. Younes, H.L.S., Simmons, R.G.: Probabilistic verification of discrete event systems using acceptance sampling. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 223–235. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45657-0_17

    Chapter  MATH  Google Scholar 

  24. Yu, M., Vasudevan, R., Johnson-Roberson, M.: Risk assessment and planning with bidirectional reachability for autonomous driving. In: 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, 31 May–31 August 2020, pp. 5363–5369. IEEE (2020). https://doi.org/10.1109/ICRA40945.2020.9197491

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jörg Hoffmann .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Baier, C. et al. (2021). Lab Conditions for Research on Explainable Automated Decisions. In: Heintz, F., Milano, M., O'Sullivan, B. (eds) Trustworthy AI - Integrating Learning, Optimization and Reasoning. TAILOR 2020. Lecture Notes in Computer Science(), vol 12641. Springer, Cham. https://doi.org/10.1007/978-3-030-73959-1_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-73959-1_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-73958-4

  • Online ISBN: 978-3-030-73959-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics