Modeling the behavior of persons with mild cognitive impairment or Alzheimer’s for intelligent environment simulation

Abstract

Intelligent environments may improve the independence and quality of life of persons with mild cognitive impairment (MCI) or Alzheimer’s disease (AD) through their ability to automatically provide assistance or guidance. In order to deploy these systems in this category of the population, it is necessary to be able to carry out validation and experiments to improve efficiency, safety, user experience and reduce installation costs. Unfortunately, this type of experiment can be difficult to perform because of the difficulty in recruiting candidates and accessing adequate intelligent environments. These problems could be partially offset with simulators. These tools can be used to simulate the behavior of an intelligent environment and its occupants in order to generate data, or to observe and evaluate their behavior. However, to design systems for populations suffering from MCI or AD, it is necessary that the simulator be able to emulate the behavior of these persons. In this paper, two approaches to simulate and generate sequences of actions containing errors usually committed by persons with this type of disease are proposed. Those approaches aim to be simple to use and both are based on the use of behavior trees. The first one consists in adding nodes to a behavior tree to simulate errors with their specific probabilities. The second approach consists in defining an interval to bind the number of errors that can be inserted through the error injection algorithm. We also present the results of the experiments carried out to evaluate these approaches. For the first experiment, several simulations were conducted and were recorded in videos. These videos were analyzed by specialists in cognitive disorders who diagnosed the avatar of these videos. The second experiment aimed at comparing the two approaches together. To do so, several action sequences were generated. The results show that our model is able to generate healthy, MCI and Alzheimer’s behaviors. The results also show that the second approach facilitates the generation of a desired number of errors.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

References

  1. Alshammari, N., Alshammari, T., Sedky, M., Champion, J., Bauer, C.: Openshs: Open smart home simulator. Sensors 17(5), 1003 (2017)

    Article  Google Scholar 

  2. Alshammari, T., Alshammari, N., Sedky, M., Howard, C.: Simadl: Simulated activities of daily living dataset. Data 3(2), 11 (2018)

    Article  Google Scholar 

  3. Anderson, J.R.: Rules of the Mind. Psychology Press, Routledge (2014)

    Book  Google Scholar 

  4. Angelucci, F., Spalletta, G., Iulio, Fd, Ciaramella, A., Salani, F., Varsi, A., Gianni, W., Sancesario, G., Caltagirone, C., Bossu, P.: Alzheimer’s disease (ad) and mild cognitive impairment (mci) patients are characterized by increased BDNF serum levels. Curr. Alzheimer Res. 7(1), 15–20 (2010)

    Article  Google Scholar 

  5. Anvari-Moghaddam, A., Monsef, H., Rahimi-Kian, A.: Optimal smart home energy management considering energy saving and a comfortable lifestyle. IEEE Trans. Smart Grid 6(1), 324–332 (2015)

    Article  Google Scholar 

  6. Aramendi, A.A., Weakley, A., Goenaga, A.A., Schmitter-Edgecombe, M., Cook, D.J.: Automatic assessment of functional health decline in older adults based on smart home data. J. Biomed. Inform. 81, 119–130 (2018). https://doi.org/10.1016/j.jbi.2018.03.009

    Article  Google Scholar 

  7. Belchior, PdC, Holmes, M., Bier, N., Bottari, C., Mazer, B., Robert, A., Kaur, N.: Performance-based tools for assessing functional performance in individuals with mild cognitive impairment. Open J. Occup. Therapy 3(3), 3 (2015)

    Article  Google Scholar 

  8. Bellagente, P., Crema, C., Depari, A., Ferrari, P., Flammini, A., Lanfranchi, G., Lenzi, G., Maddiona, M., Rinaldi, S., Sisinni, E., et al.: Remote and non-invasive monitoring of elderly in a smart city context. In: Sensors Applications Symposium (SAS), 2018 IEEE, pp. 1–6. IEEE (2018)

  9. Blender Foundation: Blender. https://www.blender.org/ January 2018 (2016)

  10. Botía, J.A., Campillo, P., Campuzano, F., Serrano, E.: UbikSim website. https://github.com/emilioserra/UbikSim/wiki. Accessed 1 June 2020 (2014)

  11. Bouchard, B., Gaboury, S., Bouchard, K., Francillette, Y.: Modeling human activities using behaviour trees in smart homes. In: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference, pp. 67–74. ACM (2018)

  12. Bouchard, K., Bouchard, B., Bouzouanea, A.: Practical guidelines to build smart homes: lessons learned. In: Opportunistic Networking, Smart Home, Smart City, Smart Systems, pp. 1–37. CRC Press, London (2014)

    Google Scholar 

  13. Bratman, M.: Intention, Plans, and Practical Reason, vol. 10. Harvard University Press, Cambridge, MA (1987)

    Google Scholar 

  14. Brush, A., Lee, B., Mahajan, R., Agarwal, S., Saroiu, S., Dixon, C.: Home automation in the wild: challenges and opportunities. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2115–2124. ACM (2011)

  15. Camilleri, G.: A generic formal plan recognition theory. In: Proceedings, 1999 International Conference on Information Intelligence and Systems, pp. 540–547. IEEE (1999)

  16. Charniak, E., Goldman, R.P.: A bayesian model of plan recognition. Artif. Intell. 64(1), 53–79 (1993)

    Article  Google Scholar 

  17. Chen, L., Hoey, J., Nugent, C.D., Cook, D.J., Yu, Z.: Sensor-based activity recognition. IEEE Trans. Syst. Man Cybern Part C (Appl. Rev.) 42(6), 790–808 (2012)

    Article  Google Scholar 

  18. Chen, L., Nugent, C.D., Biswas, J., Hoey, J.: Activity Recognition in Pervasive Intelligent Environments, vol. 4. Springer, New York (2011)

    MATH  Book  Google Scholar 

  19. Chiaravalloti, N., Goverover, Y.: Changes in the Brain: Impact on Daily Life. Springer, New York (2016)

    Google Scholar 

  20. Crawford, C.: Chris Crawford on Interactive Storytelling. New Riders, Indianapolis (2012)

    Google Scholar 

  21. Dillon, C., Serrano, C.M., Castro, D., Leguizamón, P.P., Heisecke, S.L., Taragano, F.E.: Behavioral symptoms related to cognitive impairment. Neuropsychiatr. Dis. Treat. 9, 1443 (2013)

    Article  Google Scholar 

  22. Do, H.M., Pham, M., Sheng, W., Yang, D., Liu, M.: Rish: A robot-integrated smart home for elderly care. Robot. Auton. Syst. 101, 74–92 (2018). https://doi.org/10.1016/j.robot.2017.12.008

    Article  Google Scholar 

  23. Dubois, D., Nkambou, R., Quintal, J.F., Savard, F.: Decision-making in cognitive tutoring systems. In: Advances in Intelligent Tutoring Systems, pp. 145–179. Springer, New York (2010)

    Google Scholar 

  24. Ferber, J.: Les systèmes multi-agents: vers une intelligence collective. InterEditions, Paris (1997)

    MATH  Google Scholar 

  25. Francillette, Y., Boucher, E., Bouzouane, A., Gaboury, S.: The virtual environment for rapid prototyping of the intelligent environment. Sensors 17(11), 2562 (2017)

    Article  Google Scholar 

  26. Franklin, S., Kelemen, A., McCauley, L.: Ida: A cognitive agent architecture. In: SMC’98 Conference Proceedings, 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No. 98CH36218), vol. 3, pp. 2646–2651. IEEE (1998)

  27. Franklin, S., Patterson Jr., F.: The lida architecture: adding new modes of learning to an intelligent, autonomous, software agent. Pat 703, 764–1004 (2006)

    Google Scholar 

  28. Geib, C.W., Goldman, R.P.: Partial observability and probabilistic plan/goal recognition. In: Proceedings of the International Workshop on Modeling Other Agents from Observations (MOO-05), vol. 8 (2005)

  29. Giovannetti, T., Bettcher, B.M., Brennan, L., Libon, D.J., Burke, M., Duey, K., Nieves, C., Wambach, D.: Characterization of everyday functioning in mild cognitive impairment: a direct assessment approach. Dement. Geriatr. Cogn. Disorders 25(4), 359–365 (2008)

    Article  Google Scholar 

  30. Grześ, M., Hoey, J., Khan, S.S., Mihailidis, A., Czarnuch, S., Jackson, D., Monk, A.: Relational approach to knowledge engineering for pomdp-based assistance systems as a translation of a psychological model. Int. J. Approx. Reason. 55(1), 36–58 (2014)

    Article  Google Scholar 

  31. Hallé, S., Gaboury, S., Bouchard, B.: Activity recognition through complex event processing: first findings. In: Workshops at the Thirtieth AAAI Conference on Artificial Intelligence (2016)

  32. Ho, B., Vogts, D., Wesson, J.: A smart home simulation tool to support the recognition of activities of daily living. Proc. South Afr. Inst. Comput Sci. Inf. Technol. 2019, 1–10 (2019)

    Google Scholar 

  33. Hornbæk, K., Oulasvirta, A.: What is interaction? In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 5040–5052. ACM (2017)

  34. Humphreys, G., Forde, E.: Disordered action schema and action disorganisation syndrome. Cogn. Neuropsychol. 15(6), 771–812 (1998)

    Google Scholar 

  35. Jiang, B., Fei, Y.: Smart home in smart microgrid: a cost-effective energy ecosystem with intelligent hierarchical agents. IEEE Trans. Smart Grid 6(1), 3–13 (2015)

    Article  Google Scholar 

  36. Kautz, H.A.: Reasoning About Plans. chap. A Formal Theory of Plan Recognition and Its Implementation, pp. 69–124. Morgan Kaufmann Publishers Inc., San Francisco, CA (1991)

    Google Scholar 

  37. Kormányos, B., Pataki, B.: Multilevel simulation of daily activities: Why and how? In: 2013 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), pp. 1–6. IEEE (2013)

  38. Lee, J.W., Cho, S., Liu, S., Cho, K., Helal, S.: Persim 3D: Context-driven simulation and modeling of human activities in smart spaces. IEEE Trans. Autom. Sci. Eng. 12(4), 1243–1256 (2015)

    Article  Google Scholar 

  39. Lee, J.W., Helal, A.S., Sung, Y., Cho, K.: Context activity selection and scheduling in context-driven simulation. In: Proceedings of the Symposium on Theory of Modeling & Simulation-DEVS Integrative, p. 9. Society for Computer Simulation International (2014)

  40. Leitner, G.: The Future Home is Wise. Not Smart. Springer, New York (2015)

    Book  Google Scholar 

  41. Liu, S., Helal, S., Lee, J.W.: Activity playback modeling for smart home simulation. In: International Conference on Smart Homes and Health Telematics, pp. 92–102. Springer (2015)

  42. Lotfi, A., Langensiepen, C., Mahmoud, S.M., Akhlaghinia, M.J.: Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behaviour. J. Ambient Intell. Humaniz. Comput. 3(3), 205–218 (2012)

    Article  Google Scholar 

  43. Luke, S., Cioffi-Revilla, C., Panait, L., Sullivan, K., Balan, G.: Mason: a multiagent simulation environment. Simulation 81(7), 517–527 (2005)

    Article  Google Scholar 

  44. Marcotte, R., Hamilton, H.J.: Behavior trees for modelling artificial intelligence in games: a tutorial. Comput. Games J. 6(3), 171–184 (2017). https://doi.org/10.1007/s40869-017-0040-9

    Article  Google Scholar 

  45. Marzinotto, A., Colledanchise, M., Smith, C., Ogren, P.: Towards a unified behavior trees framework for robot control. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5420–5427. IEEE (2014)

  46. McKhann, G.M., Knopman, D.S., Chertkow, H., Hyman, B.T., Jack Jr., C.R., Kawas, C.H., Klunk, W.E., Koroshetz, W.J., Manly, J.J., Mayeux, R., et al.: The diagnosis of dementia due to alzheimer’s disease: recommendations from the national institute on aging-alzheimer’s association workgroups on diagnostic guidelines for alzheimer’s disease. Alzheimer’s & Dement. 7(3), 263–269 (2011)

    Article  Google Scholar 

  47. Morris, M.E., Adair, B., Miller, K., Ozanne, E., Hansen, R., Pearce, A.J., Santamaria, N., Viega, L., Long, M., Said, C.M.: Smart-home technologies to assist older people to live well at home. J. Aging Sci. 1(1), 1–9 (2013)

    Google Scholar 

  48. Nazerfard, E., Das, B., Holder, L.B., Cook, D.J.: Conditional random fields for activity recognition in smart environments. In: Proceedings of the 1st ACM International Health Informatics Symposium, pp. 282–286. ACM (2010)

  49. Paternò, F.: Concurtasktrees: an engineered notation for task models. In: The Handbook of Task Analysis for Human–Computer Interaction, pp. 483–503. Lawrence Erlbaum Associates, Mahwah (2004)

    Google Scholar 

  50. Paternò, F., Mancini, C., Meniconi, S.: Concurtasktrees: a diagrammatic notation for specifying task models. In: Human–Computer Interaction INTERACT’97, pp. 362–369. Springer, New York (1997)

    Google Scholar 

  51. Patterson, D.J., Fox, D., Kautz, H., Philipose, M.: Fine-grained activity recognition by aggregating abstract object usage. In: Proceedings. Ninth IEEE International Symposium on Wearable Computers, 2005, pp. 44–51. IEEE (2005)

  52. Pavithra, D., Balakrishnan, R.: Iot based monitoring and control system for home automation. In: 2015 Global Conference on Communication Technologies (GCCT), pp. 169–173. IEEE (2015)

  53. Schmitter-Edgecombe, M., Parsey, C.M.: Assessment of functional change and cognitive correlates in the progression from healthy cognitive aging to dementia. Neuropsychology 28(6), 881 (2014)

    Article  Google Scholar 

  54. Schneider, M.: Resource-aware plan recognition in instrumented environments. Doctoral Thesis (2010). https://doi.org/10.22028/D291-25968

  55. Serna, A., Pigot, H., Rialle, V.: Modeling the progression of Alzheimer’s disease for cognitive assistance in smart homes. User Model. User Adapt. Interact. 17(4), 415–438 (2007)

    Article  Google Scholar 

  56. Serrano, E., Botia, J.: Validating ambient intelligence based ubiquitous computing systems by means of artificial societies. Inf. Sci. 222, 3–24 (2013). https://doi.org/10.1016/j.ins.2010.11.012

    Article  Google Scholar 

  57. Synnott, J., Nugent, C., Jeffers, P.: Simulation of smart home activity datasets. Sensors 15(6), 14162–14179 (2015)

    Article  Google Scholar 

  58. Unity Technologies: Unity—game engine. https://unity3d.com. January 2018 (2016)

  59. Van der Mussele, S., Mariën, P., Saerens, J., Somers, N., Goeman, J., De Deyn, P.P., Engelborghs, S.: Behavioral syndromes in mild cognitive impairment and alzheimer’s disease. J. Alzheimer’s Dis. 38(2), 319–329 (2014)

    Article  Google Scholar 

  60. Weiss, G.: Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. MIT Press, Cambridge (1999)

    Google Scholar 

  61. Wobcke, W.: Two logical theories of plan recognition. J. Logic Comput. 12(3), 371–412 (2002)

    MathSciNet  MATH  Article  Google Scholar 

  62. Wooldridge, M.: An Introduction to Multiagent Systems. Wiley, New York (2009)

    Google Scholar 

Download references

Acknowledgements

The Natural Sciences and Engineering Research Council of Canada (NSERC) funds and supports our research projects. Nathalie Bier is supported by the Fonds de recherche du Québec- Santé.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yannick Francillette.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Francillette, Y., Boucher, E., Bier, N. et al. Modeling the behavior of persons with mild cognitive impairment or Alzheimer’s for intelligent environment simulation. User Model User-Adap Inter 30, 895–947 (2020). https://doi.org/10.1007/s11257-020-09266-4

Download citation

Keywords

  • Simulation
  • Simulation of activity of daily life
  • Error simulation
  • Intelligent environment
  • Mild cognitive impairment
  • Alzheimer