Skip to main content
SpringerLink
Log in

Software Architecture for Smart Emotion Recognition and Regulation of the Ageing Adult

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

This paper introduces the architecture of an emotion-aware ambient intelligent and gerontechnological project named “Improvement of the Elderly Quality of Life and Care through Smart Emotion Regulation”. The objective of the proposal is to find solutions for improving the quality of life and care of the elderly who can or want to continue living at home by using emotion regulation techniques. A series of sensors is used for monitoring the elderlies’ facial and gestural expression, activity and behaviour, as well as relevant physiological data. This way the older people’s emotions are inferred and recognized. Music, colour and light are the stimulating means to regulate their emotions towards a positive and pleasant mood. Then, the paper proposes a gerontechnological software architecture that enables real-time, continuous monitoring of the elderly and provides the best-tailored reactions of the ambience in order to regulate the older person’s emotions towards a positive mood. After describing the benefits of the approach for emotion recognition and regulation in the elderly, the eight levels that compose the architecture are described.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bachorik JP, Bangert M, Loui P, Larke K, Berger J, Roew R, et al. Emotion in motion: investigating the time-course of emotional judgments of musical stimuli. Music Percept. 2009;26:355–64.

    Article  Google Scholar 

  2. Bower GH. Mood and memory. Am Psychol. 1981;36:129–48.

    Article  CAS  PubMed  Google Scholar 

  3. Brewer D, Doughtie EB. Induction of mood and mood shift. J Clin Psychol. 1980;36:215–26.

    Article  CAS  PubMed  Google Scholar 

  4. Castillo JC, Fernández-Caballero A, Castro-González Á, Salichs MA, López MT. A framework for recognizing and regulating emotions in the elderly. Lect Notes Comput Sci. 2014;8868:320–7.

    Article  Google Scholar 

  5. Chen M, Gonzalez S, Vasilakos A, Cao H, Leung VCM. Body area networks: a survey. Mob Netw Appl. 2011;16:171–93.

    Article  Google Scholar 

  6. Devillers L, Lamel L. Emotion detection in a task-oriented dialogs. In: Proceedings of IEEE international conference on multimedia & expo, Baltimore, MD, USA; July 2003, vol 3, pp 549–552.

  7. Ekman P, Friesen WV, Hager JC. Facial action coding system [e-book]. Salt Lake City: Research Nexus; 2002.

    Google Scholar 

  8. Elliot AJ, Maier MA. Color and psychological functioning. Curr Dir Psychol Sci. 2007;16:250–4.

    Article  Google Scholar 

  9. Esposito A, Fortunati L, Lugano G. Modeling emotion, behavior and context in socially believable robots and ICT interfaces. Cogn Comput. 2014;6:623–7.

    Article  Google Scholar 

  10. FaceSense. 2013. http://affect.media.mit.edu/projects.php?id=2523. Accessed 29 July 2015.

  11. Fernández-Caballero A, López MT, Saiz-Valverde S. Dynamic stereoscopic selective visual attention (DSSVA): integrating motion and shape with depth in video segmentation. Expert Syst Appl. 2008;34:1394–402.

    Article  Google Scholar 

  12. Fernández-Caballero A, Latorre JM, Pastor JM, Fernández-Sotos A. Improvement of the elderly quality of life and care through smart emotion regulation. Lect Notes Comput Sci. 2014;8868:348–55.

    Article  Google Scholar 

  13. Fernández-Caballero A, Castillo JC, López MT, Serrano-Cuerda J, Sokolova MV. INT3-Horus framework for multispectrum activity interpretation in intelligent environments. Expert Syst Appl. 2013;40:6715–27.

    Article  Google Scholar 

  14. Fernández-Sotos A, Fernández-Caballero A, Latorre JM. Elicitation of emotions through music: the influence of note value. Artif Comput Biol Med 2015;9107:488–497.

    Article  Google Scholar 

  15. García-Rodríguez C, Martínez-Tomás R, Cuadra-Troncoso JM, Rincón M, Fernández-Caballero A. A simulation tool for monitoring elderly who suffer from disorientation in a smart home. Expert Syst. 2015;32:676–87.

    Article  Google Scholar 

  16. Gascueña JM, Fernández-Caballero A. On the use of agent technology in intelligent, multi-sensory and distributed surveillance. Knowl Eng Rev. 2011;26:191–208.

    Article  Google Scholar 

  17. Gehrig T, Ekenel HK. A common framework for real-time emotion recognition and facial action unit detection. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition workshops, Colorado Springs, USA, June 2011; pp. 1–6.

  18. Gross JJ, Barrett LF. Emotion generation and emotion regulation: one or two depends on your point of view. Emot Rev. 2011;3:8–16.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Gross JJ, Levenson RW. Emotion elicitation using films. Cogn Emot. 1995;9:87–108.

    Article  Google Scholar 

  20. Jovanov E, Milenkovic A. Body area networks for ubiquitous healthcare applications: opportunities and challenges. J Med Syst. 2011;35:1245–54.

    Article  PubMed  Google Scholar 

  21. Kim JB, ParkJS OhYW. Speaker-characterized emotion recognition using online and iterative speaker adaptation. Cogn Comput. 2012;4:398–408.

    Article  Google Scholar 

  22. Kim J, André E. Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell. 2008;30:2067–83.

    Article  PubMed  Google Scholar 

  23. Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): affective ratings of pictures and instruction manual. Technical report A–8, University of Florida, USA, 2009.

  24. Lee H, Choi YS, Lee S, Park IP. Towards unobtrusive emotion recognition for affective social communication. In: Proceedings of IEEE consumer communications and networking conference, Las Vegas, NV, USA, January 2012; pp. 260–264.

  25. Livingstone SR, Palmer C, Schubert E. Emotion response to musical repetition. Emotion. 2012;12:552–67.

    Article  PubMed  Google Scholar 

  26. Livingstone RS, Thompson WF. The emergence of music from the theory of mind. Musicae Scientiae. 2009;17:83–115.

    Article  Google Scholar 

  27. López JM, Gil R, García R, Cearreta I, Garay N. Towards an ontology for describing emotions. Lect Notes Comput Sci. 2008;5288:96–104.

    Article  Google Scholar 

  28. Lozano-Monasor E, López MT, Fernández-Caballero A, Vigo-Bustos F. Facial expression recognition from webcam based on active shape models and support vector machines. Ambient Assist Living Daily Act 2014;8868:147–154.

    Google Scholar 

  29. Martínez-Rodrigo A, Zangróniz R, Pastor JM, Latorre JM, Fernández-Caballero A. Emotion detection in ageing adults from physiological sensors. Ambient Intell Softw Appl 2015;376:253–261.

    Google Scholar 

  30. McRae K, Misra S, Prasad AK, Pereira SC, Gross JJ. Bottom-up and top-down emotion generation: implications for emotion regulation. Soc Cogn Affect Neurosci. 2012;7:253–62.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Medjahed H, Istrate D, Boudy J, Baldinger JL. A pervasive multi-sensor data fusion for smart home healthcare monitoring. In: Proceedings of IEEE international conference in fuzzy systems, Taipei, Taiwan, June 2011; pp. 1466–1473.

  32. Niedenthal PM, Halberstadt JB, Setterlund MC. Being happy and seing happy: emotional state mediates visual word recognition. Cogn Emot. 1997;11:403–32.

    Article  Google Scholar 

  33. Ortiz-García-Cervigón V, Sokolova MV, García-Muñoz RM, Fernández-Caballero A. Led strips for color- and illumination-based emotion regulation at home. Ambient assisted living. Development and testing of ICT-based solutions in real life situations 2015;9455:277–287.

  34. Pail G, Huf W, Pjrek E, Winkler D, Willeit M, Praschak-Rieder N, et al. Bright-light therapy in the treatment of mood disorders. Neuropsychobiology. 2011;64:152–62.

    Article  CAS  PubMed  Google Scholar 

  35. Pavón J, Gómez-Sanz JJ, Fernández-Caballero A, Valencia-Jiménez JJ. Development of intelligent multi-sensor surveillance systems with agents. Robot Auton Syst. 2008;55:892–903.

    Article  Google Scholar 

  36. Picard RW, Vyzas E, Healey J. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Patterns Anal Mach Intell. 2001;23:1175–91.

    Article  Google Scholar 

  37. Sadri F. Ambient intelligence: a survey. ACM Comput Surv (CSUR). 2011;43(4):36.

    Article  Google Scholar 

  38. Salovey P, Rothman AJ, Detweiler JB, Steward WT. Emotional states and physical health. Am Psychol. 2000;55:110–21.

    Article  CAS  PubMed  Google Scholar 

  39. de Santos A, Sánchez C, Guerra J, Bailador G. A stress-detection system based on physiological signals and fuzzy logic. IEEE Trans Industr Electron. 2011;58:4857–65.

    Article  Google Scholar 

  40. Schachter S, Singer JE. Cognitive, social and physiological determinants of emotional state. Psychol Rev. 1962;69:379–99.

    Article  CAS  PubMed  Google Scholar 

  41. Scherer K. What are emotions? And how can they be measured? Soc Sci Inf. 2005;44:695–729.

    Article  Google Scholar 

  42. Silva PRD, Osano M, Marasinghe A, Madurapperuma AP. Towards recognizing emotion with affective dimensions through body gestures. In: Proceedings of 7th IEEE international conference on automatic face and gesture recognition, Southampton, United Kingdom, April 2006; pp. 269–274.

  43. Sokolova MV, Fernández-Caballero A, López MT, Martínez-Rodrigo A, Zangróniz R, Pastor JM. A distributed architecture for multimodal emotion identification. Trends Pract Appl Agents Multi Agent Syst Sustain 2015;372:125–132.

    Article  Google Scholar 

  44. Sokolova MV, Fernández-Caballero A, Ros L, Fernández-Aguilar L, Latorre JM. Experimentation on emotion regulation with single-colored images. Ambient assisted living. Development and testing of ICT-based solutions in real life situations 2015;9455:265–276.

  45. Soleymani M, Lichtenauer J, Pun T, Pantic M. A multi-modal affective database for affect recognition and implicit tagging. IEEE Trans Affect Comput. 2012;3:42–55.

    Article  Google Scholar 

  46. Susskind JM, Littlewort G, Bartlett MS, Movellan J, Anderson AK. Human and computer recognition of facial expressions of emotion. Neuropsychologia. 2007;45:152–62.

    Article  CAS  PubMed  Google Scholar 

  47. Velten E. A laboratory task for induction of mood states. Behav Res Ther. 1968;6:473–82.

    Article  PubMed  Google Scholar 

  48. Vogt T, André E, Wagner J. Automatic recognition of emotions from speech: a review of the literature and recommendations for practical realisation. Lect Notes Comput Sci. 2008;4868:75–91.

    Article  Google Scholar 

  49. Wagner J, André E, Jung F. Smart sensor integration: a framework for multimodal emotion recognition in real-time. In: Proceedings of 3rd international conference on affective computing and intelligent interaction and workshops, Amsterdam, The Netherlands, September 2009; pp. 1–8.

  50. Zhou J, Yu C, Riekki J, Kärkkäinen E. AmE framework: a model for emotion aware ambient intelligence. In: Proceedings of the second international conference on affective computing and intelligent interaction (ACII2007): Doctoral Consortium, September 2007.

Download references

Acknowledgments

This work was partially supported by Spanish Ministerio de Economía y Competitividad/FEDER under TIN2013-47074-C2-1-R grant. José Carlos Castillo was partially supported by a grant from Iceland, Liechtenstein and Norway through the EEA Financial Mechanism, operated by Universidad Complutense de Madrid.

Author information

Authors and Affiliations

  1. Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    José Carlos Castillo & Antonio Fernández-Caballero

  2. Robotics Lab, Universidad Carlos III de Madrid, 28911, Madrid, Spain

    José Carlos Castillo, Álvaro Castro-González & Miguel A. Salichs

  3. Instituto de Investigación en Discapacidades Neurológicas, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    José Miguel Latorre

  4. Instituto de Tecnologías Audiovisuales, Universidad de Castilla-La Mancha, 16071, Cuenca, Spain

    José Manuel Pastor

  5. Facultad de Educación de Albacete, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    Alicia Fernández-Sotos

  6. Conservatorio Profesional de Música Maestro Gómez Villa, 30530, Cieza, Murcia, Spain

    Alicia Fernández-Sotos

Authors
  1. José Carlos Castillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Álvaro Castro-González
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Fernández-Caballero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José Miguel Latorre
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. José Manuel Pastor
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alicia Fernández-Sotos
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Miguel A. Salichs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José Carlos Castillo.

Ethics declarations

Conflict of Interest

José Carlos Castillo, Álvaro Castro-González, Antonio Fernández-Caballero, José Miguel Latorre, José Manuel Pastor, Alicia Fernández-Sotos and Miguel A. Salichs declare that they have no conflict of interest.

Informed Consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Additional informed consent was obtained from all patients for which identifying information is included in this article.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by the any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Castillo, J.C., Castro-González, Á., Fernández-Caballero, A. et al. Software Architecture for Smart Emotion Recognition and Regulation of the Ageing Adult. Cogn Comput 8, 357–367 (2016). https://doi.org/10.1007/s12559-016-9383-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-016-9383-y

Keywords

Search

Navigation