Skip to main content
View expanded cover

Robotics Research pp 75–84Cite as

How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics

Part of the Springer Proceedings in Advanced Robotics book series (SPAR,volume 10)

Abstract

Endowing robots with the capability of assessing risk and making risk-aware decisions is widely considered a key step toward ensuring safety for robots operating under uncertainty. But, how should a robot quantify risk? A natural and common approach is to consider the framework whereby costs are assigned to stochastic outcomes–an assignment captured by a cost random variable. Quantifying risk then corresponds to evaluating a risk metric, i.e., a mapping from the cost random variable to a real number. Yet, the question of what constitutes a “good” risk metric has received little attention within the robotics community. The goal of this paper is to explore and partially address this question by advocating axioms that risk metrics in robotics applications should satisfy in order to be employed as rational assessments of risk. We provide instantiations of the class of risk metrics that satisfy these axioms (referred to as distortion risk metrics) and also discuss pitfalls of commonly used risk metrics in robotics that do not satisfy these axioms. Our hope is that the ideas presented here will lead to a foundational framework for quantifying risk (and hence safety) in robotics applications.

Keywords

  • Planning and decision making under uncertainty
  • Risk metrics
  • Safety

Anirudha Majumdar—Work performed as a postdoctoral scholar at Stanford University.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-28619-4_10
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   299.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-28619-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   379.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1

References

  1. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety (2016). arXiv:1606.06565

  2. Artzner, P., Delbaen, F., Eber, J.-M., Heath, D.: Coherent measures of risk. Math. Financ. 9(3), 203–228 (1999)

    MathSciNet  CrossRef  Google Scholar 

  3. Föllmer, H., Schied, A.: Stochastic Finance: An Introduction in Discrete Time. Walter de Gruyter (2011)

    Google Scholar 

  4. García, J., Fernández, F.: A comprehensive survey on safe reinforcement learning. J. Mach. Learn. Res. 16(1), 1437–1480 (2015)

    MathSciNet  MATH  Google Scholar 

  5. Glover, K., Doyle, J.C.: Relations between H\(_\infty \) and risk sensitive controllers. Analysis and Optimization of Systems. Springer, Berlin (1987)

    Google Scholar 

  6. Kuindersma, S., Grupen, R., Barto, A.: Variable risk control via stochastic optimization. Int. J. Robot. Res. 32(7), 806–825 (2013)

    CrossRef  Google Scholar 

  7. Maccheroni, F., Marinacci, M., Rustichini, A., Taboga, M.: Portfolio selection with monotone mean-variance preferences. Math. Finan. 19(3), 487–521 (2009)

    MathSciNet  CrossRef  Google Scholar 

  8. Majumdar, A., Pavone, M.: How should a robot assess risk? Towards an axiomatic theory of risk in robotics. Extended version of ISRR 2017 paper (2017). arXiv:1710.11040

  9. Majumdar, A., Singh, S., Mandlekar, A., Pavone, M.: Risk-sensitive inverse reinforcement learning via coherent risk models. Robotics Science and Systems (2017)

    Google Scholar 

  10. Mannor, S., Tsitsiklis, J.N.: Mean-variance optimization in Markov decision processes. In: International Conference on Machine Learning (2011)

    Google Scholar 

  11. B.C. on Banking Supervision: Fundamental Review of the Trading Book: A Revised Market Risk Framework. Bank for International Settlements (2014)

    Google Scholar 

  12. Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–41 (2000)

    CrossRef  Google Scholar 

  13. Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on stochastic programming: Modeling and theory. SIAM, 2nd edn. (2014)

    Google Scholar 

  14. Wang, S.: A class of distortion operators for pricing financial and insurance risks. J. Risk Insur. 67(1), 15–36 (2000)

    CrossRef  Google Scholar 

  15. Whittle, P.: Risk-sensitive linear/quadratic/gaussian control. Adv. Appl. Probab. 13(4), 764–777 (1981)

    MathSciNet  CrossRef  Google Scholar 

Download references

Acknowledgements

The authors were partially supported by the Office of Naval Research, Science of Autonomy Program, under Contract N00014-15-1-2673.

Author information

Authors and Affiliations

  1. Department of Mechanical and Aerospace Engineering, Princeton University, Princeton, NJ, 08544, USA

    Anirudha Majumdar

  2. Department of Aeronautics and Astronautics, Stanford University, Stanford, CA, 94305, USA

    Marco Pavone

Authors
  1. Anirudha Majumdar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Pavone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anirudha Majumdar .

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA

    Nancy M. Amato

  2. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

    Greg Hager

  3. Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA

    Shawna Thomas

  4. Department of Electrical Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile

    Miguel Torres-Torriti

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Majumdar, A., Pavone, M. (2020). How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics. In: Amato, N., Hager, G., Thomas, S., Torres-Torriti, M. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 10. Springer, Cham. https://doi.org/10.1007/978-3-030-28619-4_10

Download citation