Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Pervasive Systems, Algorithms and Networks

I-SPAN 2019: Pervasive Systems, Algorithms and Networks pp 15–28Cite as

A Reinforcement Learning Based Intelligent System for the Healthcare Treatment Assistance of Patients with Disabilities

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1080))

Abstract

Nowadays, one of the clinical challenge is the realization of personalized treatments, which falls into the more general paradigm of the precision medicine. On the other hand, over the last years we have assisted the rising of technologies able to assist people at home during their daily activities. In this paper we present an intelligent system, which is able to self-adapt to user’s skills aiming at assisting her/him in the healthcare treatment. The system adopts the Reinforcement Learning paradigm to adapt the way to communicate with the patient. By this way, in case of patients with physical disabilities (e.g. auditory or visual impairments) or cognitive disabilities (e.g. mild cognitive impairments), the system automatically searches for the most effective way to communicate and remind the daily treatment plan to the patient.

The AMICO project has received funding from the National Programs (PON) of the Italian Ministry of Education, Universities and Research (MIUR): code ARS01_00900 (Decree n.1989, 26 July 2018).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Paragliola, G., Coronato, A.: A reinforcement-learning-based approach for the planning of safety strategies in AAL applications. In: Intelligent Environments 2018 - Workshop Proceedings of the 14th International Conference on Intelligent Environments, Rome, Italy, 25–28 June 2018, pp. 498–505 (2018)

    Google Scholar 

  2. Ross, C.K., Steward, C.A., Sinacore, J.M.: The importance of patient preferences in the measurement of health care satisfaction. Med. Care, 31, 1138–1149 (1993). JSTOR

    Article  Google Scholar 

  3. Rao, G.N., Prasad, L.V.: How can we improve patient care? Community Eye Health 15(41), 1–3 (2002). NCBI-PMC

    Google Scholar 

  4. Patel, V.L., et al.: The coming of age of artificial intelligence in medicine. Artif. Intell. Med. 46(1), 5–17 (2009). AIME 2007

    Google Scholar 

  5. Ling, Y., et al.: Diagnostic inferencing via improving clinical concept extraction with deep reinforcement learning: a preliminary study. In: Proceedings of the 2nd Machine Learning for Healthcare Conference, pp. 271–285 (2017)

    Google Scholar 

  6. Ling, Y., et al.: Learning to diagnose: assimilating clinical narratives using deep reinforcement learning. In: IJCNLP (2017)

    Google Scholar 

  7. Schleidgen, S., Klingler, C., Bertram, T., Rogowski, W.H., Marckmann, G.: What is personalized medicine: sharpening a vague term based on a systematic literature review. BMC Med. Ethics 14, 55 (2013)

    Article  Google Scholar 

  8. Petersen, B.K.: Precision medicine as a control problem: using simulation and deep reinforcement learning to discover adaptive, personalized multi-cytokine therapy for sepsis, CoRR, abs/1802.10440 (2018)

    Google Scholar 

  9. Sahba, F., Tizhoosh, H.R., Salama, M.M.A.: A reinforcement learning framework for medical image segmentation. In: The 2006 IEEE International Joint Conference on Neural Network Proceedings (2006)

    Google Scholar 

  10. Maicas, G., Carneiro, G., Bradley, A.P., Nascimento, J.C., Reid, I.: Deep reinforcement learning for active breast lesion detection from DCE-MRI. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 665–673. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_76

    Chapter  Google Scholar 

  11. Kearns, M.J., Litman, D.J., Singh, S.P., Walker, M.A.: Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System, CoRR, abs/1106.0676 (2011)

    Google Scholar 

  12. Odom, P., Natarajan, S.: Active advice seeking for inverse reinforcement learning. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, AAMAS 2016, pp. 512–520 (2016)

    Google Scholar 

  13. Paragliola, G., Naeem, M.: Risk management for nuclear medical department using reinforcement learning. J. Reliab. Intell. Environ. 5(2), 105–113 (2019)

    Article  Google Scholar 

  14. Coronato, A., Paragliola, G., Naeem, M., De Pietro, G.: Reinforcement learning-based approach for the risk management of e-health environments: a case study. In: 14th International Conference on Signal Image Technology & Internet Based Systems, Las Plamas Spain, 26–29 November 2019

    Google Scholar 

  15. Prasad, N., Cheng, L.-F., Chivers, C., Draugelis, M., Engelhardt, B.E.: A reinforcement learning approach to weaning of mechanical ventilation in intensive care units, CoRR, abs/1704.06300 (2017)

    Google Scholar 

  16. Pilarski, P.M., Dawson, M.R., Degris, T., Fahimi, F., Carey, J.P., Sutton, R.S.: Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–7 (2011)

    Google Scholar 

  17. Reinkensmeyer, D.J., Guigon, E., Maier, M.A.: A computational model of use-dependent motor recovery following a stroke: optimizing corticospinal activations via reinforcement learning can explain residual capacity and other strength recovery dynamics. Neural Netw. Off. J. Int. Neural Netw. Soc. 29, 60–69 (2012)

    Article  Google Scholar 

  18. Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Netw. 21(4), 682–697 (2008)

    Article  Google Scholar 

  19. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall Press, Upper Saddle River (2009)

    MATH  Google Scholar 

  20. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  21. Lavori, P.W., Dawson, R.: Dynamic treatment regimes: practical design considerations. Clin. Trials 1(1), 9–20 (2004)

    Article  Google Scholar 

  22. Chakraborty, B., Moodie, E.E.: Statistical Methods for Dynamic Treat-ment Regimes. Springer, New York (2013). https://doi.org/10.1007/978-1-4614-7428-9

    Book  Google Scholar 

  23. Arjas, E., Saarela, O.: Optimal dynamic regimes: presenting a case for predictive inference. Int. J. Biostat. 6(2) (2010)

    Google Scholar 

  24. Robins, J.M.: Optimal structural nested models for optimal sequential decisions. In: Lin, D.Y., Heagerty, P.J. (eds.) Proceedings of the Second Seattle Symposium in Biostatistics, vol. 179, pp. 189–326. Springer, New York (2004). https://doi.org/10.1007/978-1-4419-9076-1_11

    Chapter  Google Scholar 

  25. Thall, P.F., Wathen, J.K.: Covariate-adjusted adaptive randomization in a sarcoma trial with multi-stage treatments. Stat. Med. 24, 1947–1964 (2005)

    Article  MathSciNet  Google Scholar 

  26. Murphy, S.A., Oslin, D.W., Rush, A.J., Zhu, J.: Methodological challenges in constructing effective treatment sequences for chronic psychiatric disorders. Neuropsychopharmacology 32, 257–262 (2006)

    Article  Google Scholar 

  27. Polydoros, A.S., Nalpantidis, L.: Survey of model-based reinforcement learning: applications on robotics. J. Intell. Robot. Syst. 86(2), 153–173 (2017)

    Article  Google Scholar 

  28. Konda, V.R., Tsitsiklis, J.N.: Actor-critic algorithms. In: Advances in Neural Information Processing (2000)

    Google Scholar 

  29. Patacchiola, M.: A Developmental Model of Trust in Humanoid Robot, University of Plymouth (2018)

    Google Scholar 

  30. Mynatt, E.D., Essa, I., Rogers, W.: Increasing the opportunities for aging in place. In: Proceedings on the 2000 Conference on Universal Usability, CUU 2000 (2000)

    Google Scholar 

  31. Kautz, H., Arnstein, L., Borriello, G., Etzioni, O., Fox, D.: An overview of the assisted cognition project. In: Proceedings on the: Conference on Universal Usability, Caregiver: The Role of Intelligent Technology in Elder Care, Edmonton (2000)

    Google Scholar 

  32. Pineau, J., Montemerlo, M., Pollack, M., Roy, N., Thrun, S.: Towards robotic assistants in nursing homes: challenges and results. Robot. Auton. Syst. 42, 271–281 (2003)

    Article  MATH  Google Scholar 

  33. Mozer, M.C.: Lessons from an adaptive home. Smart Environ. 12, 271–294 (2005)

    Article  Google Scholar 

  34. Hoey, J., Poupart, P., von Bertoldi, A., Craig, T., Boutilier, C., Mihailidis, A.: Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process. Comput. Vis. Image Underst. 144(5), 503–519 (2010)

    Article  Google Scholar 

  35. Mihailidis, A., Fernie, G.R., Barbenel, J.C.: The use of artificial intelligence in the design of an intelligent cognitive orthosis for people with dementia. Assist. Technol. 13(1), 23–29 (2001)

    Article  Google Scholar 

  36. Suresh, H., Hunt, N., Johnson, A.E.W., Celi, L.A., Szolovits, P., Ghassemi, M.: Clinical Intervention Prediction and Understanding using Deep Networks, CoRR (2017)

    Google Scholar 

  37. Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19, 780–785 (1997)

    Article  Google Scholar 

  38. Oliver, N., Garg, A., Horvitz, E.: Layered representations for learning and inferring office activity from multiple sensory channels. Comput. Vis. Image Underst. 96, 163–180 (2004)

    Article  Google Scholar 

  39. Llorca, D.F., Vilarino, F., Zhou, J., Lacey, G.: A multi-class SVM classifier for automatic hand washing quality assessment. In: Proceedings of the British Machine Vision Conference (2007)

    Google Scholar 

  40. Llorca, D.F., Parra, I., Sotelo, M.Á., Lacey, G.: A vision-based system for automatic hand washing quality assessment. Mach. Vis. Appl. 22(2), 219–234 (2011)

    Article  Google Scholar 

  41. Russell, S.J., Norvig, P., Canny, J.F., Malik, J.M., Edwards, D.D.: Artificial Intelligence: A Modern Approach. Upper Saddle River, Prentice Hall (2003)

    Google Scholar 

  42. Sutton, R.S., Barto, A.G., Reinforcement Learning: An Introduction, 2nd edn., in progress, p. 2. The MIT Press Cambridge (2014, 2015)

    Google Scholar 

  43. Bakhouya, M., Campbell, R., Coronato, A., de Pietro, G., Ranganathan, A.: Introduction to special section on formal methods in pervasive computing. ACM Trans. Auton. Adapt. Syst. (TAAS) 7(1), 6 (2012)

    Google Scholar 

  44. Cinque, M., Coronato, A., Testa, A.: Dependable services for mobile health monitoring systems. Int. J. Ambient. Comput. Intell. (IJACI) 4(1), 1–15 (2012)

    Article  Google Scholar 

  45. Coronato, A., De Pietro, G., Sannino, G.: Middleware services for pervasive monitoring elderly and ill people in smart environments. In: 2010 Seventh International Conference on Information Technology: New Generations (2010)

    Google Scholar 

  46. Cinque, M., Coronato, A., Testa, A.: A failure modes and effects analysis of mobile health monitoring systems. In: Elleithy, K., Sobh, T. (eds.) Innovations and Advances in Computer, Information, Systems Sciences, and Engineering, vol. 152, pp. 569–582. Springer, New York (2012). https://doi.org/10.1007/978-1-4614-3535-8_48

    Chapter  Google Scholar 

  47. Coronato, A., De Pietro, G., Gallo, L.: An agent based platform for task distribution in virtual environments. J. Syst. Arch. 54(9), 877–882 (2008)

    Article  Google Scholar 

  48. Coronato, A., De Florio, V., Bakhouya, M., Di Marzo Serugendo, G.: Formal modeling of socio-technical collective adaptive systems. In: 2012 IEEE Sixth International Conference on Self-Adaptive and Self-Organizing Systems Workshops, pp. 187–192 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ICAR-CNR Naples, Naples, Italy

    Antonio Coronato

  2. Universita’ Degli Studi di Napoli ‘Parthenope’, Naples, Italy

    Muddasar Naeem

Authors
  1. Antonio Coronato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muddasar Naeem
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muddasar Naeem .

Editor information

Editors and Affiliations

  1. University of Naples Federico II, Naples, Napoli, Italy

    Christian Esposito

  2. Soongsil University, Seoul, Korea (Republic of)

    Jiman Hong

  3. The University of Texas at San Antonio, San Antonio, TX, USA

    Kim-Kwang Raymond Choo

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Coronato, A., Naeem, M. (2019). A Reinforcement Learning Based Intelligent System for the Healthcare Treatment Assistance of Patients with Disabilities. In: Esposito, C., Hong, J., Choo, KK. (eds) Pervasive Systems, Algorithms and Networks. I-SPAN 2019. Communications in Computer and Information Science, vol 1080. Springer, Cham. https://doi.org/10.1007/978-3-030-30143-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30143-9_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30142-2

  • Online ISBN: 978-3-030-30143-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation