Skip to main content

Advertisement

SpringerLink
Log in

Guidelines for reinforcement learning in healthcare

  • Comment
  • Published:

From Nature Medicine

View current issue Submit your manuscript

In this Comment, we provide guidelines for reinforcement learning for decisions about patient treatment that we hope will accelerate the rate at which observational cohorts can inform healthcare practice in a safe, risk-conscious manner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1: Sequential decision-making tasks.

Debbie Maizels/Springer Nature

Fig. 2: Effective sample size in off-policy evaluation.

Debbie Maizels/Springer Nature

References

  1. Obermeyer, Z. & Emanuel, E. J. N. Engl. J. Med. 375, 1216 (2016).

    Article  Google Scholar 

  2. Parbhoo, S., Bogojeska, J., Zazzi, M., Roth, V. & Doshi-Velez, F. AMIA Summits on Translational Science Proceedings 2017, 239 (2017).

    PubMed Central  Google Scholar 

  3. Guez, A., Vincent, R. D., Avoli, M. & Pineau, J. Treatment of epilepsy via batch-mode reinforcement learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence 1671–1678 (AAAI, 2008).

  4. Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. & Faisal, A. Nat. Med. 24, 1716–1720 (2018).

  5. Chakraborty, B., Moodie, E. & Erica, E. M. Statistical Methods for Dynamic Treatment Regimes (Springer, New York, 2013).

  6. Simpson, N., Lamontagne, F. & Shankar-Hari, M. Curr Opin Crit Care. 23, 561–566 (2017).

    Article  Google Scholar 

  7. Johansson, F., Shalit, U. & Sontag, D. Learning representations for counterfactual inference. In Proceedings of the 33th International Conference on Machine Learning (ICML, 2016).

  8. Precup, D., Sutton, R. S. & Singh, S. P. Eligibility traces for off-policy policy evaluation. In Proceedings of the Seventeenth International Conference on Machine Learning 759–766 (ICML, 2000).

  9. Gottesman, O. et al. Evaluating Reinforcement Learning Algorithms in Observational Health Settings. Preprint at https://arxiv.org/abs/1805.12298 (2018).

  10. Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at https://arxiv.org/abs/1702.08608 (2017).

Download references

Author information

Author notes

  1. These authors contributed equally: Omer Gottesman, Fredrik Johansson.

Authors and Affiliations

  1. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA

    Omer Gottesman & Finale Doshi-Velez

  2. Institute for Medical Engineering and Science, MIT, Cambridge, MA, USA

    Fredrik Johansson & David Sontag

  3. Laboratory for Computational Physiology, Harvard-MIT Health Sciences & Technology, MIT, Cambridge, MA, USA

    Matthieu Komorowski & Leo Anthony Celi

  4. Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK

    Matthieu Komorowski

  5. Department of Bioengineering, Imperial College London, London, UK

    Aldo Faisal

  6. Department of Computing, Imperial College London, London, UK

    Aldo Faisal

  7. Data Science Institute, London, UK

    Aldo Faisal

  8. MRC London Institute of Clinical Sciences, London, UK

    Aldo Faisal

  9. Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA

    Leo Anthony Celi

  10. MIT Critical Data, Cambridge, MA, USA

    Leo Anthony Celi

Authors
  1. Omer Gottesman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fredrik Johansson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthieu Komorowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aldo Faisal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Sontag
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Finale Doshi-Velez
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Leo Anthony Celi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leo Anthony Celi.

Ethics declarations

Competing interests

A.A.F. has received funding from Fresenius-KABI in the past.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gottesman, O., Johansson, F., Komorowski, M. et al. Guidelines for reinforcement learning in healthcare. Nat Med 25, 16–18 (2019). https://doi.org/10.1038/s41591-018-0310-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-018-0310-5

  • Springer Nature America, Inc.

This article is cited by

Search

Navigation