Journal on Multimodal User Interfaces

, Volume 7, Issue 1–2, pp 93–109 | Cite as

A french corpus of audio and multimodal interactions in a health smart home

  • Anthony Fleury
  • Michel Vacher
  • François Portet
  • Pedro Chahuara
  • Norbert Noury
Original Paper


Health Smart Homes are nowadays a very explored research area due to the needs for automation and telemedicine to support people in loss of autonomy and also due to the evolution of the technology that led in cheap and efficient sensors. However, collecting data in this area is still very challenging. As a consequence, many studies cannot be validated on real data. In this paper, we present two realistic datasets acquired in a fully equipped Health Smart Home. The first is related to distress detection from speech (450 recorded sentences) and involved 10 participants, the second involved 15 participants who were performing several instances of seven activities of daily living (16 h of multimodal data).


Health smart home Environmental sensors Audio and speech analysis Activity monitoring Activity recognition 


  1. 1.
    Aggarwal J, Ryoo M (2011) Human activity analysis: a review. ACM Comput Surv 43:1–43CrossRefGoogle Scholar
  2. 2.
    Badii A, Boudy J (2009) CompanionAble—integrated cognitive assistive& domotic companion robotic systems for ability& security. In: Proceedings of the first congress of the Société Française des Technologies pour l’Autonomie et de Gérontechnologie (SFTAG’09), pp 18–20, TroyesGoogle Scholar
  3. 3.
    Callejas Z, López-Cózar R (2009) Designing smart home interfaces for the elderly. SIGACCESS Newsletter 95Google Scholar
  4. 4.
    Chahuara P, Portet F, Vacher M (2011) Fusion of audio and temporal multimodal data by spreading activation for dweller localisation in a smart home. In: STAMI, Space, Time and Ambient Intelligence, pp 17–22. Barcelona, SpainGoogle Scholar
  5. 5.
    Chan M, Estève D, Escriba C, Campo E (2008) A review of smart homes—present state and future challenges. Comput Methods Programs Biomed 91(1):55–81CrossRefGoogle Scholar
  6. 6.
    Cook DJ, Schmitter-Edgecombe M (2009) Assessing the quality of activities in a smart environment. Methods of Information in Medicine 48(5):480–485.
  7. 7.
    Cornet G, Carré M (2008) Technologies pour le soin, l’autonomie et le lien social des personnes âgées : quoi de neuf ? Gérontologie et société 126:113–128CrossRefGoogle Scholar
  8. 8.
    Coutaz J, Crowley JL, Dobson S, Garlan D (2005) Context is key. Commun ACM 48(3):49–53CrossRefGoogle Scholar
  9. 9.
    Demiris G, Rantz M, Aud M, Marek K, Tyrer H, Skubic M, Hussam A (2004) Older adults’ attitudes towards and perceptions of “smart home” technologies: a pilot study. Med Inform Internet Med 29(2):87–94CrossRefGoogle Scholar
  10. 10.
    Dey AK (2001) Understanding and using context. Personal Ubiquitous Comput 5(1):4–7CrossRefGoogle Scholar
  11. 11.
    Filho G, Moir TJ (2010) From science fiction to science fact: a smart-house interface using speech technology and a photo-realistic avatar. Int J Comput Appl Technol 39(8):32–39CrossRefGoogle Scholar
  12. 12.
    Fleury A, Noury N, Vacher M (2009) A wavelet-based pattern recognition algorithm to classify postural transition in humans. In: 17th European signal processing conference (EUSIPCO 2009), pp. 2047–2051. Glasgow, ScotlandGoogle Scholar
  13. 13.
    Fleury A, Noury N, Vacher M (2011) Improving supervised classification of activities of daily living using prior knowledge. Int J E-Health Med Commun 2(1):17–34CrossRefGoogle Scholar
  14. 14.
    Fleury A, Vacher M, Noury N (2010) SVM-based multi-modal classification of activities of daily living in health smart homes: Sensors, algorithms and first experimental results. IEEE Trans Inf Technol Biomed 14(2):274–283CrossRefGoogle Scholar
  15. 15.
    Gödde F, Möller S, Engelbrecht KP, Kühnel C, Schleicher R, Naumann A, Wolters M (2008) Study of a speech-based smart home system with older users. In: International workshop on intelligent user interfaces for ambient assisted, living, pp 17–22Google Scholar
  16. 16.
    Hamill M, Young V, Boger J, Mihailidis A (2009) Development of an automated speech recognition interface for personal emergency response system. J NeuroEng Rehabil 26(6)Google Scholar
  17. 17.
    Health Smart Home Datasets.
  18. 18.
    Hornbrook M, Stevens V, Wingfield D, Hollis J, Greenlick M, Ory M (1994) Preventing falls among community-dwelling older persons: results from a randomized trial. Gerontologist 34(1):16–23CrossRefGoogle Scholar
  19. 19.
    Intille S, Larson K, Tapia E, Beaudin J, Kaushik P, Nawyn J, Rockinson R (2006) Using a live-in laboratory for ubiquitous computing research. In: Fishkin K, Schiele B, Nixon P, Quigley A (eds) Pervasive computing. Lecture notes in computer science, vol 3968, chap. 22, pp 349–365. Springer, Berlin. doi: 10.1007/11748625.22
  20. 20.
    Kang MS, Kim KM, Kim HC (2006) A questionnaire study for the design of smart homes for the elderly. In: Proceedings of healthcom, pp 265–268Google Scholar
  21. 21.
    van Kasteren T, Noulas A, Englebienne G, Kröse B (2008) Accurate activity recognition in a home setting. In: UbiComp’08: proceedings of the 10th international conference on ubiquitous computing, pp 1–9. ACM, New York. doi: 10.1145/1409635.1409637
  22. 22.
    Katz S, Akpom C (1976) A measure of primary sociobiological functions. Int J Health Serv 6(3):493–508CrossRefGoogle Scholar
  23. 23.
    Koskela T, Väänänen-Vainio-Mattila K (2004) Evolution towards smart home environments: empirical evaluation of three user interfaces. Personal Ubiquitous Comput 8:234–240CrossRefGoogle Scholar
  24. 24.
    Lawton M, Brody E (1969) Assesment of older people: self-maintaining and instrumental activities of daily living. Gerontologist 9:179–186CrossRefGoogle Scholar
  25. 25.
    Le Bellego G, Noury N, Virone G, Mousseau M, Demongeot J (2006) A model for the measurement of patient activity in a hospital suite. IEEE Trans Inf Technol Biomed 10(1):92–99CrossRefGoogle Scholar
  26. 26.
    Lecouteux B, Vacher M, Portet F (2011) Distant speech recognition in a smart home: comparison of several multisource asrs in realistic conditions. In: Interspeech 2011, pp 2273–2276. Florence, ItalyGoogle Scholar
  27. 27.
    López-Cózar R, Callejas Z (2010) Multimodal dialogue for ambient intelligence and smart environments. In: Nakashima H, Aghajan H, Augusto JC (eds) Handbook of ambient intelligence and smart environments, pp 559–579. Springer, BerlinGoogle Scholar
  28. 28.
    Mäyrä F, Soronen A, Vanhala J, Mikkonen J, Zakrzewski M, Koskinen I, Kuusela K (2006) Probing a proactive home: challenges in researching and designing everyday smart environments. Human Technol 2:158–186Google Scholar
  29. 29.
    Meyer S, Rakotonirainy A (2003) A survey of research on context-aware homes. In: ACSW, Frontiers, pp 159–168Google Scholar
  30. 30.
    Moncrieff S, Venkatesh S, West GAW (2007) Dynamic privacy in a smart house environment. In: IEEE multimedia and expo, pp 2034–2037Google Scholar
  31. 31.
    Niessen ME, van Maanen L, Andringa TC (2008) Disambiguating sounds through context. In: Proceedings of the second IEEE international conference on semantic computing, ICSC2008, pp 88–95. IEEE Computer SocietyGoogle Scholar
  32. 32.
    Noury N, Hervé T, Rialle V, Virone G, Mercier E (2000) Monitoring behavior in home using a smart fall sensor and position sensors. In: IEEE-EMBS microtechnologies in medicine& biology, pp 607–610Google Scholar
  33. 33.
    Noury N, Poujaud J, Fleury A, Nocua R, Haddidi T, Rumeau P (2011) Activity recognition in pervasive intelligent environments, Atlantis ambient and pervasive intelligence, vol 4, chap Smart Sweet Home: a pervasive environment for sensing our daily activity, p 328. Atlantis PressGoogle Scholar
  34. 34.
    Portet F, Fleury A, Vacher M, Noury N (2009) Determining useful sensors for automatic recognition of activities of daily living in health smart home. In: Intelligent data international workshop on analysis in medicine and pharmacology (IDAMAP2009), pp 63–64. Verona, ItalyGoogle Scholar
  35. 35.
    Portet F, Vacher M, Golanski C, Roux C, Meillon B (2011) Design and evaluation of a smart home voice interface for the elderly—acceptability and objection aspects. Personal and Ubiquitous Computing, pp 1–30. doi: 10.1007/s00779-011-0470-5 (in press)
  36. 36.
    Rialle V, Rumeau P, Cornet G, Franco A (2007) Les gérontechnologies: au cœur de l’innovation hospitalière et médico-sociale. Techniques hospitalières 703:53–58Google Scholar
  37. 37.
    Sharkey A, Sharkey N (2012) Granny and the robots: ethical issues in robot care for the elderly. Ethics Inf Technol 1–14 (in press)Google Scholar
  38. 38.
    Vacher M, Fleury A, Portet F, Serignat JF, Noury N (2010) New developments in biomedical engineering, chap. complete sound and speech recognition system for health smart homes: application to the recognition of activities of daily living, pp 645–673. Intech BookGoogle Scholar
  39. 39.
    Vacher M, Fleury A, Serignat JF, Noury N, Glasson H (2008) Preliminary evaluation of speech/sound recognition for telemedicine application in a real environment. In: Proceedings of interspeech 2008, pp 496–499. Brisbane, AustraliaGoogle Scholar
  40. 40.
    Vacher M, Portet F, Fleury A, Noury N (2011) Development of audio sensing technology for ambient assisted living: applications and challenges. Int J E-Health Med Commun 2(1):35–54CrossRefGoogle Scholar
  41. 41.
    Van-Thinh V, Bremond F, Thonnat M (2003) Automatic video interpretation: a novel algorithm for temporal scenario recognition. In: Proceedings of the 18th international joint conference on artificial intelligence, pp 1295–1300. Morgan Kaufmann Publishers Inc., Acapulco, MexicoGoogle Scholar
  42. 42.
    Weiser M (1991) The computer for the 21st century. Sci Am 265(3):66–75CrossRefGoogle Scholar
  43. 43.
    Youngblood GM, Cook DJ (2007) Data mining for hierarchical model creation. IEEE Trans Syst Man Cybern Part C 37(4):561–572CrossRefGoogle Scholar
  44. 44.
    Ziefle M, Wilkowska W (2010) Technology acceptability for medical assistance. In: 4th international conference on pervasive computing technologies for healthcare (PervasiveHealth). Munich, Germany Google Scholar
  45. 45.
    Zouba N, Bremond F, Thonnat M, Anfosso A, Pascual E, Mallea P, Mailland V, Guerin O (2009) A computer system to monitor older adults at home: preliminary results. Gerontechnol J 8(3):129–139Google Scholar

Copyright information

© OpenInterface Association 2012

Authors and Affiliations

  • Anthony Fleury
    • 1
    • 2
  • Michel Vacher
    • 3
  • François Portet
    • 3
  • Pedro Chahuara
    • 3
  • Norbert Noury
    • 4
  1. 1.Univ. Lille Nord de FranceLilleFrance
  2. 2.Mines Douai, IADouaiFrance
  3. 3.Laboratoire d’Informatique de GrenobleGrenobleFrance
  4. 4.Univ. Lyon, lab. INL, UMR CNRS/UCBL/ECL/INSA 5270VilleurbanneFrance

Personalised recommendations