On Applying Ambient Intelligence to Assist People with Profound Intellectual and Multiple Disabilities

  • Michal KosiedowskiEmail author
  • Arkadiusz Radziuk
  • Piotr Szymaniak
  • Wojciech Kapsa
  • Tomasz Rajtar
  • Maciej Stroinski
  • Carmen Campomanes-Alvarez
  • B. Rosario Campomanes-Alvarez
  • Mitja Lustrek
  • Matej Cigale
  • Erik Dovgan
  • Gasper Slapnicar
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1038)


Advances in ambient intelligence technologies achieved over recent years allow building ICT systems capable of supporting people in executing previously difficult tasks. This includes providing new opportunities to people with special needs, such as people with disabilities. In this paper we discuss our approach at designing and developing a smart platform that can assist people with profound intellectual and multiple disabilities (PIMD) in achieving a portion of independence. We apply this approach in the INSENSION project executed within the Horizon 2020 research and innovation programme of the European Commission. In this paper, we describe the characteristics of the disability in question, focusing on the main challenge, which is the inability of individuals with PIMD to use symbols in their interaction. Due to the fact that, to our best knowledge, the topic of constructing a system that could assist this interaction has not been undertaken so far, we were required to thoroughly analyze how individuals with PIMD interact using non-symbolic behaviors. We then defined requirements for the platform capable of supporting their interaction with other people and possibly living environment, designed the architecture and built the first version with the use of AI methods. The evaluation of this version confirmed the soundness of our approach, which enables us to continue our work towards successful implementation of the planned ambient intelligence system and its validation in real-life scenarios.


Smart assistive technologies Non-symbolic interaction Accessible technologies 



The research presented herewith has been conducted within the INSENSION project which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement no. 780819.


  1. 1.
    Berndt, R.-D., Takenga, M.C., Kuehn, S., Preik, P., Berndt, S., Brandstoetter, M., Planinc, R., Kampel, M.: An assisted living system for the elderly – FEARLESS concept. In: Proceedings of the IADIS Multi Conference on Computer Science and Information Systems eHealth2012, Lisbon, pp. 131–138 (2012)Google Scholar
  2. 2.
    Azevedo, C., Chesta, C., Coelho, J., Dimola, D., Duarte, C., Manca, M., Nordvik, J., Paterno, F., Sanders, A.-M., Santoro, C.: Towards a platform for persuading older adults to adopt healthy behaviors. In: Orji, R., Reisinger, M., Busch, M.D., A., Kaptein, M., Mattheiss, E. (eds.) Proceedings of the Personalization in Persuasive Technology Workshop, Persuasive Technology 2017, Amsterdam (2017)Google Scholar
  3. 3.
    Cvetković, B., Gjoreski, M., Šorn, J., Frešer, M., Bogdański, M., Jackowska, K., Kosiedowski, M., Luštrek, M.: Management of physical, mental and environmental stress at the workplace. In: International Conference on Intelligent Environments (IE), pp. 76–83. IEEE (2017)Google Scholar
  4. 4.
    Andò, B., Baglio, S., La Malfa, S., Marletta, V.: Innovative smart sensing solutions for the visually impaired. In: Pereira, J. (ed.) Handbook of Research on Personal Autonomy Technologies and Disability Informatics, pp. 60–74. IGI Global, Hershey (2011)CrossRefGoogle Scholar
  5. 5.
    Praveen, N., Karanth, N., Megha, M.S.: Sign language interpreter using a smart glove. In: International Conference on Advances in Electronics, Computers and Communications (ICAECC 2014), pp. 1–5. IEEE (2014)Google Scholar
  6. 6.
    Atkin, K., Lorch, M.P.: An ecological method for the sampling of nonverbal signalling behaviours of young children with profound and multiple learning disabilities (PMLD). Dev. Neurorehabil. 19(4), 211–225 (2016)Google Scholar
  7. 7.
    Rotter, B., Kane, G., Gallé, B.: Nichtsprachliche Kommunikation: Erfassung und Förderung. Geistige Behinderung 31, 1–26 (1992)Google Scholar
  8. 8.
    Hall, S.S., Arron, K., Sloneem, J., Oliver, C.: Health and sleep problems in Cornelia de Lange syndrome: a case control study. J. Intellect. Disabil. Res. 52(5), 458–468 (2008)CrossRefGoogle Scholar
  9. 9.
    Vos, P., de Cock, P., Munde, V., Petry, K., van den Noortgate, W., Maes, B.: The tell-tale: what do heart rate, skin temperature and skin conductance reveal about emotions of people with severe and profound intellectual disabilities? Res. Dev. Disabil. 33(4), 1117–1127 (2012)CrossRefGoogle Scholar
  10. 10.
    Rowland, C., Fried-Oken, M.: Communication matrix: a clinical and research assessment tool targeting children with severe communication disorders. J. Pediatr. Rehabil. Med. 3, 319–329 (2010)Google Scholar
  11. 11.
    Ross, E., Oliver, C.: Preliminary analysis of the psychometric properties of the Mood, Interest and Pleasure Questionnaire (MIPQ) for adults with severe and profound learning disabilities. Br. J. Clin. Psychol. 42, 81–93 (2003)CrossRefGoogle Scholar
  12. 12.
    Breau, L.M., McGrath, P.J., Camfield, C.S., Finley, A.G.: Psychometric properties of the non-communicating children’s pain checklist-revised. Pain 99, 349–357 (2002)CrossRefGoogle Scholar
  13. 13.
    Lotan, M., Moe-Nilssen, R., Ljunggren, A.E., Strand, L.I.: Reliability of the Non-Communicating Adult Pain Checklist (NCAPC), assessed by different groups of health workers. Res. Dev. Disabil. 30, 735–745 (2009)CrossRefGoogle Scholar
  14. 14.
    Regnard, C., Reynolds, J., Watson, B., Matthews, D., Gibson, L., Clarke, C.: Understanding distress in people with severe communication difficulties: developing and assessing the Disability Distress Assessment Tool (DisDAT). J. Intellect. Disabil. Res. 51(4), 277–292 (2007)CrossRefGoogle Scholar
  15. 15.
    ELAN (Version 5.2) (Computer software). Max Planck Institute for Psycholinguistics, Nijmegen. Accessed 15 Jan 2019
  16. 16.
    Brennan, K. (ed.): A Guide to the Business Analysis Body of Knowledger. Iiba (2009)Google Scholar
  17. 17.
    Ekman, P., Rosenberg, E.L. (eds.): What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, Oxford (1997)Google Scholar
  18. 18.
    OpenPose: Multi-Person Pose Estimation (Computer software). Carnegie Mellon, The Robotics Institute. Accessed 15 Jan 2019
  19. 19.
    Howard, A.: Elementary Linear Algebra. Wiley, Hoboken (2010)Google Scholar
  20. 20.
    Hwang, B.W., Kim, S., Lee, S.W.: A full-body gesture database for automatic gesture recognition. In: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (FGR 2006), pp. 243–248. IEEE (2006)Google Scholar
  21. 21.
    Theodorou, T., Mporas I., Fakotakis N.: Audio feature selection for recognition of non-linguistic vocalization sounds. In: Hellenic Conference on Artificial Intelligence, pp. 395–405. Springer, Cham (2014)CrossRefGoogle Scholar
  22. 22.
    Truong, K.P., van Leeuwen, D.A.: Automatic discrimination between laughter and speech. Speech Commun. 49(2), 144–158 (2007)CrossRefGoogle Scholar
  23. 23.
    Weninger, F., Schuller, B., Wollmer, M., Rigoll, G.: Localization of non-linguistic events in spontaneous speech by non-negative matrix factorization and long short-term memory. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5840–5843. IEEE (2011)Google Scholar
  24. 24.
    Swarnkar, V., Abeyratne, U.R., Amrulloh, Y., Hukins, C., Triasih, R., Setyati, A.: Neural network based algorithm for automatic identification of cough sounds. In: 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1764–1767. IEEE (2013)Google Scholar
  25. 25.
    Amoh, J., Odame, K.: Deep neural networks for identifying cough sounds. IEEE Trans. Biomed. Circuits Syst. 10(5), 1003–1011 (2016)CrossRefGoogle Scholar
  26. 26.
    Munde, V., Vlaskamp, C., Vos, P., Maes, B., Ruijssenaars, W.: Physiological measurements as validation of alertness observations: an exploratory case study of three individuals with profound intellectual and multiple disabilities. Intellect. Dev. Disabil. 50(4), 300–310 (2012)CrossRefGoogle Scholar
  27. 27.
    Lima, M., Silva, K., Amaral, I., Magalhães, A., De Sousa, L.: Beyond behavioural observations: a deeper view through the sensory reactions of children with profound intellectual and multiple disabilities. Child: Care Health Dev. 39(3), 422–431 (2013)Google Scholar
  28. 28.
    Vos, P., De Cock, P., Petry, K., Van Den Noortgate, W., Maes, B.: Do you know what I feel? A first step towards a physiological measure of the subjective well-being of persons with profound intellectual and multiple disabilities. J. Appl. Res. Intellect. Disabil. 23(4), 366–378 (2010)CrossRefGoogle Scholar
  29. 29.
    Heusch, G., Anjos, A., Marcel, S.: A reproducible study on remote heart rate measurement. arXiv preprint arXiv:1709.00962 (2017)
  30. 30.
    Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3430–3437 (2013)Google Scholar
  31. 31.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: 1981 Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  32. 32.
    Michelson, B.M.: Event-driven architecture overview. Patricia Seybold Group 2(12), 10–1571 (2006)Google Scholar
  33. 33.
    Node-RED: Flow-based programming for the Internet of Things (Computer software). JS Foundation. Accessed 15 Jan 2019

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Michal Kosiedowski
    • 1
    Email author
  • Arkadiusz Radziuk
    • 1
  • Piotr Szymaniak
    • 1
  • Wojciech Kapsa
    • 1
  • Tomasz Rajtar
    • 1
  • Maciej Stroinski
    • 1
  • Carmen Campomanes-Alvarez
    • 2
  • B. Rosario Campomanes-Alvarez
    • 2
  • Mitja Lustrek
    • 3
  • Matej Cigale
    • 3
  • Erik Dovgan
    • 3
  • Gasper Slapnicar
    • 3
  1. 1.Poznan Supercomputing and Networking CenterPoznanPoland
  2. 2.CTIC Technological CentreGijonSpain
  3. 3.Jozef Stefan InstituteLjubljanaSlovenia

Personalised recommendations