Personal and Ubiquitous Computing

, Volume 23, Issue 1, pp 145–157 | Cite as

Recognition of audible disruptive behavior from people with dementia

  • Jessica BeltránEmail author
  • René Navarro
  • Edgar Chávez
  • Jesús Favela
  • Valeria Soto-Mendoza
  • Catalina Ibarra
Original Article


Frequently, people with dementia exhibit abnormal behaviors that may cause self-injury or burden their caregivers. Some audible manifestations of these problematic behaviors are of vocal nature (e.g., shouting, mumbling, or cursing), others are environmental sounds (e.g., tapping or slamming). The timely detection of these behaviors could enact non-pharmacological interventions which in turn can assist caregivers or prevent escalation of the disruption with other fellow residents in nursing homes. We conducted a field study in a geriatric residence to gather naturalistic data. With the participation of five residents for 203 h of observation and of the 242 incidents of problematic behaviors were registered, 85% of them had a distinctive auditory manifestation. We used a combination of standard speech detection techniques, along with a novel environmental sound recognition methodology based on the entropy of the signal. We conducted experiments using realistic data, i.e., audio immersed in natural background noise. Based on classification results with F1 score above 87%, we conclude that audible cues can be used to enact non-pharmacological interventions aimed at reducing problematic behaviors, or mitigating their negative impact.



  1. 1.
    Alberdi A, Aztiria A, Basarab A (2016) On the early diagnosis of alzheimer’s disease from multimodal signals: a survey. Artif Intell Med 71:1–29. Google Scholar
  2. 2.
    Beck C, Richards K, Lambert C, Doan R, Landes RD, Whall A, Algase D, Kolanowski A, Feldman Z (2011) Factors associated with problematic vocalizations in nursing home residents with dementia. Gerontologist 51(3):389–405Google Scholar
  3. 3.
    Beltrán J, Chávez E, Favela J (2015) Scalable identification of mixed environmental sounds, recorded from heterogeneous sources. Pattern Recogn Lett 68:153–160Google Scholar
  4. 4.
    Beltrán J., Navarro R, Chávez E., Favela J, Soto-Mendoza V, Ibarra C (2014) Detecting disruptive vocalizations for ambient assisted interventions for dementia. In: Ambient assisted living and daily activities, Springer, pp 356–363Google Scholar
  5. 5.
    Bronkhorst AW (2000) The cocktail party phenomenon: a review of research on speech intelligibility in multiple-talker conditions. Acta Acustica united with Acustica 86(1):117–128 . (2000-01-01T00:00:00). Google Scholar
  6. 6.
    Burns K, Jayasinha R, Tsang R, Brodaty H (2012) Behaviour management: a guide to good practice dementia collaborative research centre - assessment and better careGoogle Scholar
  7. 7.
    Camarena-Ibarrola A, Chávez E, Tellez ES (2009) Progress in pattern recognition, image analysis, computer vision, and applications: 14th Iberoamerican conference on pattern recognition, CIARP 2009, Guadalajara, Jalisco, Mexico, November 15-18, 2009. Proceedings, chap. Robust Radio Broadcast Monitoring Using a Multi-Band Spectral Entropy Signature. Springer, Berlin, pp 587–594Google Scholar
  8. 8.
    Chen F, Adcock J, Krishnagiri S (2008) Audio privacy: reducing speech intelligibility while preserving environmental sounds. In: Proceedings of the 16th ACM international conference on multimedia, MM ’08. ACM, New York, pp 733–736
  9. 9.
    Cohen-Mansfield J (1997) Conceptualization of agitation: results based on the cohen-mansfield agitation inventory and the agitation behavior mapping instrument. Int Psychogeriatr 8(S3):309–315Google Scholar
  10. 10.
    Cohen-Mansfield J (1999) Assessment of agitation in older adults. Wiley, HobokenGoogle Scholar
  11. 11.
    Cummings J, Mega M, Gray K, Rosenberg-Thompson S, Carusi D, Gornbein J (2014) The neuropsychiatric inventory: comprehensive assessment of psychopathology in dementia. neurology 1994; 44: 2308–2314. Dement Geriatr Cogn Disord Extra 4:131–139Google Scholar
  12. 12.
    Cummings JL, Arciniegas DB, Brooks BR, Herndon RM, Lauterbach EC, Pioro EP, Robinson RG, Scharre DW, Schiffer RB, Weintraub D (2006) Defining and diagnosing involuntary emotional expression disorder. CNS Spectr 11(S6):1–11Google Scholar
  13. 13.
    Dennis J, Tran H, Chng ES (2013) Image feature representation of the subband power distribution for robust sound event classification. IEEE Trans Audio Speech Lang Process 21(2):367–377Google Scholar
  14. 14.
    Dennis J, Tran HD, Chng ES (2013) Overlapping sound event recognition using local spectrogram features and the generalised hough transform. Pattern Recogn Lett 34(9):1085–1093Google Scholar
  15. 15.
    Desai AK, Desai FG (2014) Management of behavioral and psychological symptoms of dementia. Current Geriatrics Reports 3(4):259–272Google Scholar
  16. 16.
    Fick W, van der Borgh J, Jansen S, Koopmans R (2014) The effect of a lollipop on vocally disruptive behavior in a patient with frontotemporal dementia: a case-study. Int Psychogeriatr 26(12):2023–2026Google Scholar
  17. 17.
    Gao B, Woo WL (2014) Wearable audio monitoring: content-based processing methodology and implementation. IEEE Transactions on Human-Machine Systems 44(2):222–233Google Scholar
  18. 18.
    Gu J, Gao B, Chen Y, Jiang L, Gao Z, Ma X, Ma Y, Woo WL, Jin J (2017) Wearable social sensing: content-based processing methodology and implementation. IEEE Sensors J 17(21):7167–7176Google Scholar
  19. 19.
    von Gunten A, Favre M, Gurtner C, Abderhalden C (2011) Vocally disruptive behavior (vdb) in the institutionalized elderly: a naturalistic multiple case report. Arch Gerontol Geriatr 52(3):e110–e116Google Scholar
  20. 20.
    Hearst MA (1998) Support vector machines. IEEE Intell Syst 13(4):18–28. Google Scholar
  21. 21.
    Huang X, Baker J, Reddy R (2014) A historical perspective of speech recognition. Commun ACM 57 (1):94–103Google Scholar
  22. 22.
    Jalbert JJ, Daiello LA, Lapane KL (2008) Dementia of the alzheimer type. Epidemiol Rev 30(1):15–34Google Scholar
  23. 23.
    König A., Sacco G, Bensadoun G, Bremond F, David R, Verhey F, Aalten P, Robert P, Manera V (2015) The role of information and communication technologies in clinical trials with patients with alzheimer’s disease and related disorders. Front Aging Neurosci 7:1–5Google Scholar
  24. 24.
    Krijnders J, Niessen M, Andringa T (2010) Sound event recognition through expectancy-based evaluation ofsignal-driven hypotheses. Pattern Recogn Lett 31(12):1552–1559Google Scholar
  25. 25.
    Krishnamurthy N, Hansen JH (2009) Babble noise: modeling, analysis, and applications. IEEE Trans Audio Speech Lang Process 17(7):1394–1407Google Scholar
  26. 26.
    Liaqat D, Nemati E, Rahman M, Kuang J (2017) A method for preserving privacy during audio recordings by filtering speech. In: Life sciences conference (LSC), 2017 IEEE, IEEE, pp 79–82Google Scholar
  27. 27.
    Mahesha P, Vinod D (2012) Feature based classification of dysfluent and normal speech. In: Proceedings of the Second international conference on computational science, engineering and information technology, ACM, pp 594–597Google Scholar
  28. 28.
    Mitrović D, Zeppelzauer M, Breiteneder C (2010) Features for content-based audio retrieval. Adv Comput 78:71–150Google Scholar
  29. 29.
    Navarretta C (2014) The automatic identification of the producers of co-occurring communicative behaviours. Cogn Comput 6(4):689–698Google Scholar
  30. 30.
    Navarro RF, Rodriguez M, Favela J (2014) Intervention tailoring in augmented cognition systems for elders with dementia. IEEE Journal of Biomedical and Health Informatics 18(1):361–367Google Scholar
  31. 31.
    O’Shaughnessy D (2000) Speech communications: human and machine. Institute of Electrical and Electronics Engineers.
  32. 32.
    Oshima C, Itou N, Nishimoto K, Yasuda K, Hosoi N, Yamashita H, Nakayama K, Horikawa E (2013) A music therapy system for patients with dementia who repeat stereotypical utterances. Information and Media Technologies 8(2):605–616Google Scholar
  33. 33.
    Peintner B, Jarrold W, Vergyriy D, Richey C, Tempini MLG, Ogar J (2008) Learning diagnostic models using speech and language measures. In: Conference proceedings : ... Annual international conference of the ieee engineering in medicine and biology society. IEEE Engineering in Medicine and Biology Society. Annual Conference, pp 4648–51Google Scholar
  34. 34.
    Potamitis I, Ganchev T (2008) Multimedia services in intelligent environments: advanced tools and methodologies, chap. Generalized recognition of sound events: approaches and applications. Springer, Berlin, pp 41–79Google Scholar
  35. 35.
    Qawaqneh Z, Mallouh AA, Barkana BD (2017) Deep neural network framework and transformed mfccs for speaker’s age and gender classification. Knowl-Based Syst 115:5–14Google Scholar
  36. 36.
    Rabbitt SM, Kazdin AE, Scassellati B (2015) Integrating socially assistive robotics into mental healthcare interventions: Applications and recommendations for expanded use. Clin Psychol Rev 35:35–46Google Scholar
  37. 37.
    Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286Google Scholar
  38. 38.
    Rincon E, Beltran J, Tentori M, Favela J, Chavez E (2013) A context-aware baby monitor for the automatic selective archiving of the language of infants. In: 2013 mexican international conference on computer science (ENC), IEEE, pp 60–67Google Scholar
  39. 39.
    Sakar BE, Isenkul ME, Sakar CO, Sertbas A, Gurgen F, Delil S, Apaydin H, Kursun O (2013) Collection and analysis of a parkinson speech dataset with multiple types of sound recordings. IEEE Journal of Biomedical and Health Informatics 17(4):828–834Google Scholar
  40. 40.
    Serizel R, Bisot V, Essid S, Richard G (2018) Acoustic features for environmental sound analysis. In: Computational analysis of sound scenes and events, Springer, pp 71–101Google Scholar
  41. 41.
    Sezgin MC, Gunsel B, Kurt GK (2012) Perceptual audio features for emotion detection. EURASIP Journal on Audio, Speech, and Music Processing 2012(1):1–21Google Scholar
  42. 42.
    Smith D, Ma L, Ryan N (2006) Acoustic environment as an indicator of social and physical context. Pers Ubiquit Comput 10(4):241–254Google Scholar
  43. 43.
    Thomas C, Keselj V, Cercone N, Rockwood K, Asp E (2005) Automatic detection and rating of dementia of alzheimer type through lexical analysis of spontaneous speech. In: 2005 IEEE international conference Mechatronics and automation, vol 3. pp 1569–1574.
  44. 44.
    Wang D, Brown G (2006) Computational auditory scene analysis: principles, algorithms, and applications. Wiley-IEEE PressGoogle Scholar
  45. 45.
    Yusupov A, Galvin JE (2014) Vocalization in dementia: a case report and review of the literature. Case reports in Neurology 6(1):126–133Google Scholar
  46. 46.
    Zhuang X, Huang J, Potamianos G, Hasegawa-Johnson M (2009) Acoustic fall detection using gaussian mixture models and gmm supervectors. In: 2009. ICASSP 2009. IEEE international conference on acoustics, speech and signal processing, pp 69–72.

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.CONACYT IPN-CITEDITijuanaMexico
  2. 2.Department of Industrial EngineeringUniversity of SonoraHermosilloMexico
  3. 3.Computer Science DepartmentCenter for Scientific Research and Higher Education at Ensenada (CICESE)EnsenadaMexico
  4. 4.Research Center on Applied MathematicsAutonomous University of CoahuilaSaltilloMexico

Personalised recommendations