Skip to main content
SpringerLink
Log in

Affective Computing for Brain Health Disorders

  • Living reference work entry
  • First Online:
Handbook of Computational Neurodegeneration

Abstract

Affective computing has immense potential to benefit the treatment and care of brain health disorders. Affective computing (also referred to as artificial emotion intelligence or emotion AI) is the study and development of systems and devices that can recognize, interpret, process, and simulate emotion or other affective phenomena. Health conditions across the lifespan – including neurodevelopmental, psychiatric, and neurodegenerative conditions – have benefited by different affective computing tools and technologies. Using the latest advances in computer vision, signal processing, and pattern recognition, facial indicators, head movements and pose, body movements, gaze, and vocal indicators have been found that identify depressed patients and monitor stage and severity of depression progression. Social media behavior analysis and keystroke analysis have also been found to be useful in detecting depression and suicidal ideations. For Parkinson’s disease, objectively assessing facial expression changes and vocal impairments has been found to be beneficial to detect and monitor Parkinson’s, identify subtypes, monitor treatment responses, and differentiate between commonly confused disorders. Fine motor control, vocal impairments, and eye movement have also been found to detect and monitor early Alzheimer’s disease. There is a myriad of other possible affective computing brain health applications. Efforts are needed to ensure ethical development of affective computing applications for brain health that account for algorithmic and human bias.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Aalbers G et al (2019) Social media and depression symptoms: a network perspective. J Exp Psychol Gen 148(8):1454

    Article  Google Scholar 

  • Abbas A et al (2021) Computer vision-based assessment of motor functioning in schizophrenia: use of smartphones for remote measurement of schizophrenia symptomatology. Digital Biomarkers 5(1):29–36

    PubMed  PubMed Central  Google Scholar 

  • Ahmed S et al (2013) Connected speech as a marker of disease progression in autopsy-proven Alzheimer’s disease. Brain 136(12):3727–3737

    PubMed  PubMed Central  Google Scholar 

  • Albers MW et al (2015) At the interface of sensory and motor dysfunctions and Alzheimer’s disease. Alzheimers Dement 11(1):70–98

    PubMed  Google Scholar 

  • Alghowinem S et al (2013a) Head pose and movement analysis as an indicator of depression. In 2013 Humaine association conference on affective computing and intelligent interaction. IEEE

    Google Scholar 

  • Alghowinem S et al (2013b) Eye movement analysis for depression detection. In 2013 IEEE international conference on image processing. IEEE

    Google Scholar 

  • Alghowinem S et al (2016) Cross-cultural depression recognition from vocal biomarkers. Interspeech

    Google Scholar 

  • Ali MR et al (2020) Facial expressions can detect Parkinson’s disease: preliminary evidence from videos collected online. arXiv preprint arXiv:2012.05373

    Google Scholar 

  • Alzheimer’s Research UK (2015) Women and dementia: A marginalised majority. Alzheimer’s Research UK, Cambridge, UK

    Google Scholar 

  • Association AP (2013) Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub, Washington, DC

    Google Scholar 

  • Au-Yeung W-TM et al (2021) Monitoring behaviors of patients with late-stage dementia using passive environmental sensing approaches: a case series. Am J Geriatr Psychiatry

    Google Scholar 

  • Babulal GM et al (2016) Creating a driving profile for older adults using GPS devices and naturalistic driving methodology. F1000Research 5

    Google Scholar 

  • Beuscher LM et al (2017) Socially assistive robots: measuring older adults’ perceptions. J Gerontol Nurs 43(12):35–43

    PubMed  PubMed Central  Google Scholar 

  • Bowen LK et al (2013) Effects of Parkinson’s disease on fundamental frequency variability in running speech. J Med Speech Lang Pathol 21(3):235

    PubMed  PubMed Central  Google Scholar 

  • Cacheda F et al (2019) Early detection of depression: social network analysis and random forest techniques. J Med Internet Res 21(6):e12554

    PubMed  PubMed Central  Google Scholar 

  • Caligiuri MP, Ellwanger J (2000) Motor and cognitive aspects of motor retardation in depression. J Affect Disord 57(1–3):83–93

    CAS  PubMed  Google Scholar 

  • Chen SC, Jones C, Moyle W (2018) Social robots for depression in older adults: a systematic review. J Nurs Scholarsh 50(6):612–622

    PubMed  Google Scholar 

  • Chen Y et al (2021) Defining brain health: a concept analysis. Int J Geriatr Psychiatry. https://onlinelibrary.wiley.com/doi/abs/10.1002/gps.5564

  • Cohn JF et al (2009) Detecting depression from facial actions and vocal prosody. In 2009 3rd international conference on affective computing and intelligent interaction and workshops. IEEE

    Google Scholar 

  • Cohn JF et al (2018) Multimodal assessment of depression from behavioral signals. In The handbook of multimodal-multisensor interfaces: signal processing, architectures, and detection of emotion and cognition-volume 2, pp 375–417. Association for Computing Machinery (ACM) Books

    Google Scholar 

  • Coppersmith G et al (2018) Natural language processing of social media as screening for suicide risk. Biomed Inform Insights 10:1178222618792860

    PubMed  PubMed Central  Google Scholar 

  • Crutcher MD et al (2009) Eye tracking during a visual paired comparison task as a predictor of early dementia. Am J Alzheimers Dis Other Dement 24(3):258–266

    Google Scholar 

  • Cummins N, et al (2011) An investigation of depressed speech detection: Features and normalization. In Twelfth annual conference of the international speech communication association

    Google Scholar 

  • Cummins N et al (2015) Analysis of acoustic space variability in speech affected by depression. Speech Comm 75:27–49

    Google Scholar 

  • De Choudhury M et al (2013a) Predicting depression via social media. In Proceedings of the international AAAI conference on web and social media

    Google Scholar 

  • De Choudhury M, Counts S, Horvitz E (2013b) Social media as a measurement tool of depression in populations. In Proceedings of the 5th annual ACM web science conference

    Google Scholar 

  • De Choudhury M et al (2016) Discovering shifts to suicidal ideation from mental health content in social media. In Proceedings of the 2016 CHI conference on human factors in computing systems

    Google Scholar 

  • Dibeklioglu H, Hammal Z, Cohn JF (2018) Dynamic multimodal measurement of depression severity using deep autoencoding. IEEE J Biomed Health Inform 22(2):525–536

    PubMed  Google Scholar 

  • Eby DW et al (2012) Driving behaviors in early stage dementia: a study using in-vehicle technology. Accid Anal Prev 49:330–337

    PubMed  Google Scholar 

  • el Kaliouby R (2017) We need computers with empathy. MIT Technol Rev 120:8–9

    Google Scholar 

  • Eyre HA et al (2017) Neural correlates of apathy in late-life depression: a pilot [(18) F]FDDNP positron emission tomography study. Psychogeriatrics 17(3):186–193

    PubMed  PubMed Central  Google Scholar 

  • Eyre HA, Berk M, Lavretsky H. (2021) Convergence mental health: a transdisciplinary approach to innovation. Oxford University Press, Oxford

    Google Scholar 

  • Fagherazzi G et al (2021) Voice for health: the use of vocal biomarkers from research to clinical practice. Digital Biomarkers 5(1):78–88

    PubMed  PubMed Central  Google Scholar 

  • Fernández G et al (2016) Patients with mild Alzheimer’s disease fail when using their working memory: evidence from the eye tracking technique. J Alzheimers Dis 50(3):827–838

    PubMed  Google Scholar 

  • Guntuku SC et al (2017) Detecting depression and mental illness on social media: an integrative review. Curr Opin Behav Sci 18:43–49

    Google Scholar 

  • Hoffman-Ruddy B et al (2001) A preliminary study of the effects of sub thalamic nucleus (STN) deep brain stimulation (DBS) on voice and speech characteristics in Parkinson’s disease (PD). Clin Linguist Phon 15(1–2):97–101

    CAS  PubMed  Google Scholar 

  • Horackova K et al (2019) Prevalence of late-life depression and gap in mental health service use across European regions. Eur Psychiatry 57:19–25

    PubMed  Google Scholar 

  • Islam MR et al (2018) Depression detection from social network data using machine learning techniques. Health Inf Sci Syst 6(1):8

    PubMed  PubMed Central  Google Scholar 

  • Joshi J et al (2012) Neural-net classification for spatio-temporal descriptor based depression analysis. In Proceedings of the 21st international conference on pattern recognition (ICPR2012). IEEE

    Google Scholar 

  • Joshi J et al (2013) Can body expressions contribute to automatic depression analysis? In 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE

    Google Scholar 

  • König A et al (2015) Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimer’s Dementia Diagnosis Assess Disease Monitor 1(1):112–124

    Google Scholar 

  • Ladas A et al (2014) Eye blink rate as a biological marker of mild cognitive impairment. Int J Psychophysiol 93(1):12–16

    PubMed  Google Scholar 

  • Maddage NC et al (2009) Video-based detection of the clinical depression in adolescents. Annu Int Conf IEEE Eng Med Biol Soc 2009:3723–3726

    PubMed  Google Scholar 

  • MIT Media Lab: Affective Computing Group. 2021 [01/22/2021]. Available from: https://affect.media.mit.edu/

  • Mitchell AJ, Vaze A, Rao S (2009) Clinical diagnosis of depression in primary care: a meta-analysis. Lancet 374(9690):609–619

    PubMed  Google Scholar 

  • Mohebbi M et al (2019) Prevalence of depressive symptoms and its associated factors among healthy community-dwelling older adults living in Australia and the United States. Int J Geriatr Psychiatry 34(8):1208–1216

    PubMed  PubMed Central  Google Scholar 

  • Mundt JC et al (2012) Vocal acoustic biomarkers of depression severity and treatment response. Biol Psychiatry 72(7):580–587

    PubMed  PubMed Central  Google Scholar 

  • Norton S et al (2014) Potential for primary prevention of Alzheimer’s disease: an analysis of population-based data. Lancet Neurol 13(8):788–794

    PubMed  Google Scholar 

  • Pampouchidou A et al (2017) Automatic assessment of depression based on visual cues: a systematic review. IEEE Trans Affect Comput 10(4):445–470

    Google Scholar 

  • Parak J et al (2015) Evaluation of the beat-to-beat detection accuracy of PulseOn wearable optical heart rate monitor. In 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE

    Google Scholar 

  • Patel V (2016) Deaths registered in England and Wales: 2015. Office for National Statistics, London

    Google Scholar 

  • Picard RW (1995) Affective Computing. MIT Media Laboratory Perceptual Computing Section Technical Report No. 321. https://affect.media.mit.edu/pdfs/95.picard.pdf

  • Picard RW (2000) Affective computing. MIT Press, Cambridge, MA

    Google Scholar 

  • Picard RW (2010) Affective computing: from laughter to IEEE. IEEE Trans Affect Comput 1(1):11–17

    Google Scholar 

  • Pistono A et al (2016) Pauses during autobiographical discourse reflect episodic memory processes in early Alzheimer’s disease. J Alzheimers Dis 50(3):687–698

    PubMed  PubMed Central  Google Scholar 

  • Postuma R et al (2012) How does parkinsonism start? Prodromal parkinsonism motor changes in idiopathic REM sleep behaviour disorder. Brain 135(6):1860–1870

    CAS  PubMed  Google Scholar 

  • Scherer S et al (2013) Investigating voice quality as a speaker-independent indicator of depression and PTSD. Interspeech

    Google Scholar 

  • Scherer S et al (2014) Dyadic behavior analysis in depression severity assessment interviews. Proc ACM Int Conf Multimodal Interact 2014:112–119

    PubMed  PubMed Central  Google Scholar 

  • Scoglio AA et al (2019) Use of social robots in mental health and well-being research: systematic review. J Med Internet Res 21(7):e13322

    PubMed  PubMed Central  Google Scholar 

  • Smith KM, Williamson JR, Quatieri TF (2017) Vocal markers of motor, cognitive, and depressive symptoms in Parkinson's disease. In 2017 seventh international conference on affective computing and intelligent interaction (ACII). IEEE

    Google Scholar 

  • Smith E et al (2020a) Rebooting late-life mental health innovation and entrepreneurship with convergence science. Am J Geriatr Psychiatry 28:591–596

    PubMed  Google Scholar 

  • Smith E et al (2020b) Convergence mental health: a new pathway for transdisciplinary innovation and entrepreneurship. Psychiatric Times

    Google Scholar 

  • Sobin C, Sackeim HA (1997) Psychomotor symptoms of depression. Am J Psychiatry 154(1):4–17

    CAS  PubMed  Google Scholar 

  • Song H et al (2018) Feature attention network: Interpretable depression detection from social media. In PACLIC

    Google Scholar 

  • Stasak B et al (2021) Read speech voice quality and disfluency in individuals with recent suicidal ideation or suicide attempt. Speech Comm 132

    Google Scholar 

  • Stratou G, et al (2013) Automatic nonverbal behavior indicators of depression and PTSD: exploring gender differences. In 2013 Humaine association conference on affective computing and intelligent interaction. IEEE

    Google Scholar 

  • Trevino AC, Quatieri TF, Malyska N (2011) Phonologically-based biomarkers for major depressive disorder. EURASIP J Adv Signal Process 2011(1):1–18

    Google Scholar 

  • Tsanas A et al (2012a) Novel speech signal processing algorithms for high-accuracy classification of Parkinson’s disease. IEEE Trans Biomed Eng 59(5):1264–1271

    PubMed  Google Scholar 

  • Tsanas A et al (2012b) Using the cellular mobile telephone network to remotely monitor parkinsons disease symptom severity. IEEE Trans Biomed Eng 9

    Google Scholar 

  • UN (2019) World population ageing 2019. United Nations Department of Economic and Social Affairs

    Google Scholar 

  • Venek V et al (2017) Adolescent suicidal risk assessment in clinician-patient interaction. IEEE Trans Affect Comput 8(2):204–215

    Google Scholar 

  • Vesel C et al (2020) Effects of mood and aging on keystroke dynamics metadata and their diurnal patterns in a large open-science sample: a BiAffect iOS study. J Am Med Inform Assoc 27(7):1007–1018

    PubMed  PubMed Central  Google Scholar 

  • Williamson JR et al (2013) Vocal biomarkers of depression based on motor incoordination. In Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge

    Google Scholar 

  • Williamson JR et al 2014 Vocal and facial biomarkers of depression based on motor incoordination and timing. In Proceedings of the 4th international workshop on audio/visual emotion challenge

    Google Scholar 

  • World Health Organization (2021) Ethics and governance of artificial intelligence for health: World Health Organization guidance. World Health Organization, Geneva

    Google Scholar 

  • World Health Organization: Depression. (2020). Available from: https://www.who.int/news-room/fact-sheets/detail/depression

  • Yang Y, Fairbairn C, Cohn JF (2013) Detecting depression severity from vocal prosody. IEEE Trans Affect Comput 4(2):142–150

    PubMed  PubMed Central  Google Scholar 

  • Yeung A et al (2021) Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer’s dementia. Alzheimers Res Ther 13(1):1–10

    Google Scholar 

  • Zhong B-L et al (2019) Depressive symptoms in elderly chinese primary care patients: prevalence and sociodemographic and clinical correlates. J Geriatr Psychiatry Neurol 32(6):312–318

    PubMed  Google Scholar 

  • Zola SM et al (2013) A behavioral task predicts conversion to mild cognitive impairment and Alzheimer’s disease. Am J Alzheimers Dis Other Dement 28(2):179–184

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The PRODEO Institute, San Francisco, CA, USA

    Erin Smith & Harris A. Eyre

  2. Organisation for Economic Co-operation and Development (OECD), Paris, France

    Erin Smith & Harris A. Eyre

  3. Stanford University, Palo Alto, CA, USA

    Erin Smith

  4. Global Brain Health Institute, University of California, San Francisco, CA, USA

    Erin Smith & Harris A. Eyre

  5. Trinity College, Dublin, Ireland

    Erin Smith & Harris A. Eyre

  6. Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA

    Eric A. Storch & Harris A. Eyre

  7. Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles (UCLA), Los Angeles, CA, USA

    Helen Lavretsky

  8. Chambers-Grundy Center for Transformative Neuroscience, Department of Brain Health, School of Integrated Health Sciences, University of Nevada Las Vegas (UNLV), Las Vegas, NV, USA

    Jeffrey L. Cummings

  9. IMPACT, The Institute for Mental and Physical Health and Clinical Translation, Deakin University, Geelong, VIC, Australia

    Harris A. Eyre

Authors
  1. Erin Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric A. Storch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Helen Lavretsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeffrey L. Cummings
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Harris A. Eyre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erin Smith .

Editor information

Editors and Affiliations

  1. Department of Informatics, Ionian University Department of Informatics, Corfu, Greece

    Panayiotis Vlamos

  2. Wilfrid Laurier University, Brantford, Canada

    Ilias S. Kotsireas

  3. ResearchGate, ETH Zurich ResearchGate, Zurich, Zürich, Switzerland

    Ioannis Tarnanas

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Smith, E., Storch, E.A., Lavretsky, H., Cummings, J.L., Eyre, H.A. (2021). Affective Computing for Brain Health Disorders. In: Vlamos, P., Kotsireas, I.S., Tarnanas, I. (eds) Handbook of Computational Neurodegeneration. Springer, Cham. https://doi.org/10.1007/978-3-319-75479-6_36-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-75479-6_36-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-75479-6

  • Online ISBN: 978-3-319-75479-6

  • eBook Packages: Springer Reference Biomedicine and Life SciencesReference Module Biomedical and Life Sciences

Publish with us

Policies and ethics

Search

Navigation